Nov 04 02:31:46.499655 ip-10-0-128-201 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 02:31:46.877530 ip-10-0-128-201 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:31:46.877530 ip-10-0-128-201 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 02:31:46.877530 ip-10-0-128-201 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:31:46.877530 ip-10-0-128-201 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:31:46.880472 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880354    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.880472 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880474    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880478    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880481    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880484    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880488    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880491    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880495    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880497    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880500    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880503    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880505    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880508    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880511    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880513    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880516    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880519    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880521    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880524    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880526    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.880544 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880529    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880532    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880535    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880537    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880540    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880542    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880552    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880557    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880560    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880563    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880566    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880569    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880572    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880576    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880580    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880583    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880586    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880589    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880592    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.880989 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880594    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880597    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880599    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880602    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880605    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880607    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880610    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880613    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880616    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880618    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880621    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880623    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880626    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880628    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880631    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880634    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880638    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880640    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880643    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880646    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.881447 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880649    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880651    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880653    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880656    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880659    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880661    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880664    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880667    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880670    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880673    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880675    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880678    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880681    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880683    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880686    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880688    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880691    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880694    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880696    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880699    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.881931 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880702    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880704    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880707    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880710    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880712    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880715    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880718    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880721    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880724    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880726    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880729    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880732    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880735    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880737    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880740    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880742    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880746    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880749    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880751    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.882424 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880754    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880756    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880759    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880761    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880764    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880766    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880769    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.880771    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882125    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882133    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882136    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882140    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882143    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882146    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882149    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882152    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882155    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882167    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882170    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882173    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.882867 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882176    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882179    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882183    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882186    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882188    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882191    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882193    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882196    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882199    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882201    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882204    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882207    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882209    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882212    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882214    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882218    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882220    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882223    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882225    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882227    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.883500 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882230    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882232    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882235    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882239    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882241    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882244    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882247    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882250    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882252    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882257    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882260    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882265    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882268    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882270    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882273    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882275    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882278    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882280    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882283    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.883982 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882286    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882288    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882291    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882293    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882296    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882299    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882301    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882304    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882306    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882310    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882313    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882316    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882319    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882322    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882324    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882327    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882330    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882332    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882335    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882337    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.884452 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882341    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882343    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882346    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882349    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882352    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882355    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882357    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882360    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882363    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882365    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882381    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882383    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882386    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882389    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882391    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882394    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882397    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882399    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882402    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.884928 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882405    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882407    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882410    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882412    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882415    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882417    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882420    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882423    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882425    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882429    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882432    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882435    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882437    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882440    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882443    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882445    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882922    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882927    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882930    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.885387 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882933    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882937    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882941    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882944    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882947    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882950    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882952    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882955    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882959    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882962    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882966    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882969    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882971    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882974    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882977    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882980    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882983    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882985    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882988    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.885855 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882991    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882994    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882996    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.882999    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883002    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883004    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883007    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883010    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883013    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883015    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883018    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883021    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883024    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883026    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883029    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883031    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883034    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883036    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883039    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883042    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.886322 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883045    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883047    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883050    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883053    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883055    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883058    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883061    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883063    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883066    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883068    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883071    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883073    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883076    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883079    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883082    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883084    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883087    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883089    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883092    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883095    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.886811 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883097    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883100    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883103    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883106    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883109    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883112    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883114    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883117    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883119    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883122    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883124    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883127    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883130    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883132    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883135    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883137    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883140    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883143    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883145    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.887320 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883148    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883150    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883153    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883155    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883158    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883160    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883163    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883165    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883168    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883171    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883173    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883176    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883178    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883181    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883183    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883186    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883189    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883191    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883194    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.887787 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883196    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883199    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883201    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883204    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883207    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883210    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883303    2573 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883310    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883316    2573 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883321    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883327    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883332    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883341    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883347    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883353    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883356    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883360    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883363    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883382    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883385    2573 flags.go:64] FLAG: --cgroup-root=""
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883388    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883391    2573 flags.go:64] FLAG: --client-ca-file=""
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883394    2573 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 02:31:46.888281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883397    2573 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883401    2573 flags.go:64] FLAG: --cluster-domain=""
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883404    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883407    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883410    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883414    2573 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883418    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883421    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883424    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883427    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883430    2573 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883433    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883436    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883439    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883442    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883446    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883449    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883452    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883455    2573 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883457    2573 flags.go:64] FLAG: --enable-server="true"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883461    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883465    2573 flags.go:64] FLAG: --event-burst="100"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883468    2573 flags.go:64] FLAG: --event-qps="50"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883471    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 02:31:46.888849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883473    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883476    2573 flags.go:64] FLAG: --eviction-hard=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883480    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883483    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883486    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883489    2573 flags.go:64] FLAG: --eviction-soft=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883492    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883495    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883498    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883501    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883504    2573 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883507    2573 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883510    2573 flags.go:64] FLAG: --feature-gates=""
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883514    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883517    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883521    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883524    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883527    2573 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883530    2573 flags.go:64] FLAG: --help="false"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883533    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883536    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883539    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883544    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 02:31:46.889467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883547    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883550    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883553    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883556    2573 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883559    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883561    2573 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883564    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883567    2573 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883570    2573 flags.go:64] FLAG: --kube-reserved=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883573    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883576    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883579    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883581    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883584    2573 flags.go:64] FLAG: --lock-file=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883587    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883590    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883593    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883598    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883601    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883604    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883607    2573 flags.go:64] FLAG: --logging-format="text"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883610    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883613    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883616    2573 flags.go:64] FLAG: --manifest-url=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883618    2573 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 02:31:46.890022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883623    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883626    2573 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883630    2573 flags.go:64] FLAG: --max-pods="110"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883633    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883636    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883639    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883643    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883646    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883649    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883652    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883658    2573 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883661    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883664    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883668    2573 flags.go:64] FLAG: --pod-cidr=""
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883671    2573 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883674    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883677    2573 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883680    2573 flags.go:64] FLAG: --port="10250"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883683    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883686    2573 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0eea7a91cca9de342"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883689    2573 flags.go:64] FLAG: --qos-reserved=""
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883692    2573 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883695    2573 flags.go:64] FLAG: --register-node="true"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883698    2573 flags.go:64] FLAG: --register-with-taints=""
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883701    2573 flags.go:64] FLAG: --registry-burst="10"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883704    2573 flags.go:64] FLAG: --registry-qps="5"
Nov 04 02:31:46.890636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883707    2573 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883710    2573 flags.go:64] FLAG: --reserved-memory=""
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883714    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883717    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883720    2573 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883723    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883726    2573 flags.go:64] FLAG: --runonce="false"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883730    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883733    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883736    2573 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883739    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883741    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883746    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883749    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883752    2573 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883755    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883758    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883761    2573 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883763    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883768    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883771    2573 flags.go:64] FLAG: --system-cgroups=""
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883773    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883778    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883781    2573 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883784    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 02:31:46.891253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883788    2573 flags.go:64] FLAG: --tls-min-version=""
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883791    2573 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883794    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883797    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883800    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883803    2573 flags.go:64] FLAG: --v="2"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883806    2573 flags.go:64] FLAG: --version="false"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883810    2573 flags.go:64] FLAG: --vmodule=""
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883814    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.883817    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883946    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883950    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883954    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883956    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883959    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883962    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883965    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883968    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883970    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883974    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883977    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883979    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.891878 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883982    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883985    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883988    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883990    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883995    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.883997    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884000    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884002    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884005    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884008    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884010    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884013    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884015    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884020    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884024    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884026    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884029    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884032    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884035    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884037    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.892429 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884040    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884042    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884046    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884049    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884052    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884054    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884058    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884060    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884063    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884066    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884069    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884071    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884074    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884076    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884079    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884081    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884086    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884088    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884091    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884093    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.892911 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884096    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884099    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884102    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884104    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884107    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884110    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884112    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884115    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884117    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884120    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884122    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884125    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884128    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884130    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884133    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884135    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884138    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884140    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884143    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.893427 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884146    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884149    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884152    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884155    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884157    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884160    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884162    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884165    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884167    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884171    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884174    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884176    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884179    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884181    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884184    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884187    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884189    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884192    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884194    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884197    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.893869 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884200    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884202    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884206    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884209    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884211    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884214    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884216    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884219    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884222    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884225    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884229    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884231    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884234    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884237    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884240    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.894385 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.884266    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884301    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884304    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884307    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884310    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884313    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884316    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884319    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884322    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884325    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884327    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884330    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884333    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884335    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884338    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884341    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884343    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884346    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884348    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884351    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.894748 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884354    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884356    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884359    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884361    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884364    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884381    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884384    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884387    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884389    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884392    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884396    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884401    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884404    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884407    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884409    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884412    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884414    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884417    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884420    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.895232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884423    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884426    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884429    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884431    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884434    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884437    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884439    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884442    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884445    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884447    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884450    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884452    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884455    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884458    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884460    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884463    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884466    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884469    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884471    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884474    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.895700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884476    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884479    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884483    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884486    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884490    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884494    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884496    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884499    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884501    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884504    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884506    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884509    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884514    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884516    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884519    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884522    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884524    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884527    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884529    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884532    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.896179 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884534    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884537    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884540    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884542    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884545    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884548    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884550    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884553    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884555    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884558    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884560    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884563    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884565    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884568    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884570    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884573    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884576    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884579    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884582    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.896707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884584    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884587    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884589    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884591    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884594    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884598    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884601    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884604    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.884607    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.891136    2573 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.891152    2573 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.891165    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.891170    2573 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891225    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891230    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891234    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891237    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891240    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891243    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.897216 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891247    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891250    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891252    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891255    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891258    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891261    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891263    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891266    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891269    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891272    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891274    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891277    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891280    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891283    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891285    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891288    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891291    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891293    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891296    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891299    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.897828 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891301    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891304    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891307    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891311    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891315    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891318    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891320    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891323    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891326    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891328    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891331    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891333    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891336    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891339    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891341    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891344    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891347    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891349    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891352    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891354    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.898481 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891357    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891359    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891362    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891364    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891382    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891385    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891387    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891390    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891393    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891395    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891398    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891400    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891404    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891406    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891409    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891412    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891415    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891418    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891421    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891423    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.899232 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891427    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891431    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891434    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891437    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891440    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891443    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891445    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891449    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891452    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891455    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891458    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891460    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891463    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891465    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891468    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891470    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891473    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891476    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891478    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.899904 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891481    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891484    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891486    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891489    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891491    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891494    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891497    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891499    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891502    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891505    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891508    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891510    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891513    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891515    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891518    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891520    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891523    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891526    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891529    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.900602 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891531    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891534    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.891561    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891701    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891705    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891708    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891711    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891714    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891716    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891719    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891721    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891724    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891726    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891729    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891731    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891734    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.901044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891737    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891739    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891742    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891744    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891748    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891752    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891755    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891758    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891762    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891765    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891768    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891771    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891773    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891776    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891778    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891781    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891783    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891786    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891788    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891791    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.901556 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891793    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891796    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891799    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891801    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891804    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891806    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891810    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891814    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891817    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891820    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891823    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891826    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891829    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891831    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891834    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891837    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891839    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891842    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891845    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.902224 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891847    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891850    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891853    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891856    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891858    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891861    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891863    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891866    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891868    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891871    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891873    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891876    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891878    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891881    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891884    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891887    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891889    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891892    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891895    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891897    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.902700 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891900    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891903    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891906    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891908    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891911    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891915    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891919    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891923    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891927    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891931    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891935    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891939    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891943    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891947    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891951    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891954    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891960    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891963    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891965    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.903265 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891968    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891970    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891973    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891975    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891978    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891980    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891983    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891985    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891988    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891990    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891993    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891995    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.891998    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892000    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892003    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.892027    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892057    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:31:46.903749 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892061    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892064    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892066    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892069    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892071    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892074    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892076    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892079    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892082    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892084    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892087    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892089    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892094    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892097    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892100    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892103    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892105    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892108    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892111    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:31:46.904150 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892113    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892116    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892118    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892121    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892123    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892126    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892128    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892131    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892133    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892136    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892138    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892141    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892143    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892146    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892148    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892151    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892153    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892156    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892158    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892161    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:31:46.904703 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892163    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892166    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892168    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892171    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892173    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892177    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892180    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892182    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892185    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892187    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892190    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892193    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892195    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892198    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892200    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892202    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892205    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892207    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892210    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892212    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:31:46.905236 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892214    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892217    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892219    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892222    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892224    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892227    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892230    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892234    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892237    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892242    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892245    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892248    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892250    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892253    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892255    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892258    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892260    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892264    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892267    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:31:46.905792 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892270    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892272    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892275    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892277    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892280    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892283    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892286    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892288    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892291    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892294    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892298    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892301    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892304    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892306    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892308    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892311    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892313    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892316    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892318    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:31:46.906282 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892321    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892324    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892328    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892331    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892334    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892336    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892339    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:46.892341    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.892408    2573 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.894391    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.895238    2573 server.go:1014] "Starting client certificate rotation"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.895767    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.895807    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.896988    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:31:46.906782 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.900403    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:31:46.914457 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.914443    2573 log.go:25] "Validated CRI v1 runtime API"
Nov 04 02:31:46.917486 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.917471    2573 log.go:25] "Validated CRI v1 image API"
Nov 04 02:31:46.918290 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.918268    2573 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 02:31:46.923663 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.923642    2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7afbd5d7-5cf1-4bc9-b44d-bdce48a4d326:/dev/nvme0n1p4 bf6ab72f-58bd-4cab-a5e7-cd7bad8e30ed:/dev/nvme0n1p3]
Nov 04 02:31:46.923723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.923663    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 02:31:46.928459 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.928342    2573 manager.go:217] Machine: {Timestamp:2026-11-04 02:31:46.927290804 +0000 UTC m=+0.306174285 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3167298 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e40765fe492fe7f3032e1d976a46e SystemUUID:ec2e4076-5fe4-92fe-7f30-32e1d976a46e BootID:f1c968c5-c6c9-4b5c-8ddb-959c12646422 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:4e:ff:7e:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:4e:ff:7e:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:1c:36:08:18:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 02:31:46.928459 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.928459    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 02:31:46.928567 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.928555    2573 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 02:31:46.930966 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.930943    2573 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 02:31:46.931131 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.930968    2573 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-201.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 02:31:46.931175 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.931134    2573 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 02:31:46.931175 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.931142    2573 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 02:31:46.931175 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.931165    2573 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:31:46.934966 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.934955    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:31:46.935986 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.935977    2573 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 02:31:46.939232 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939221    2573 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 02:31:46.939348 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939339    2573 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 02:31:46.939900 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939890    2573 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 02:31:46.939929 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939905    2573 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 02:31:46.939929 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939917    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 02:31:46.939929 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939925    2573 kubelet.go:405] "Adding apiserver pod source"
Nov 04 02:31:46.940036 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.939933    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 02:31:46.940235 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.940221    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:31:46.942190 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.942170    2573 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 02:31:46.946073 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.946058    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:31:46.949071 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949053    2573 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 02:31:46.949198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949187    2573 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 02:31:46.949539 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949525    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 02:31:46.949644 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949634    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 02:31:46.949723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949715    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 02:31:46.949797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949786    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 02:31:46.949876 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949868    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 02:31:46.949944 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.949936    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 02:31:46.950040 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950016    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 02:31:46.950040 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950037    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 02:31:46.950248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950048    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 02:31:46.950248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950058    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 02:31:46.950248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950073    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 02:31:46.950248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950091    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 02:31:46.950766 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950751    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 02:31:46.950810 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.950769    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 02:31:46.954714 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.954697    2573 server.go:1263] "Started kubelet"
Nov 04 02:31:46.954799 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.954760    2573 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 02:31:46.954874 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.954823    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 02:31:46.954919 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.954889    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 02:31:46.955386 ip-10-0-128-201 systemd[1]: Started Kubernetes Kubelet.
Nov 04 02:31:46.956005 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.955908    2573 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 02:31:46.958897 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.958880    2573 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 02:31:46.960844 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:46.960826    2573 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 02:31:46.962674 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.962659    2573 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 02:31:46.962753 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.962664    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 02:31:46.962808 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:46.962755    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:31:46.963161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963143    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 02:31:46.963161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963149    2573 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 02:31:46.963298 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963170    2573 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 02:31:46.963298 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963279    2573 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 02:31:46.963298 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963287    2573 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 02:31:46.963847 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963830    2573 factory.go:55] Registering systemd factory
Nov 04 02:31:46.963847 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.963848    2573 factory.go:223] Registration of the systemd container factory successfully
Nov 04 02:31:46.964039 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964025    2573 factory.go:153] Registering CRI-O factory
Nov 04 02:31:46.964039 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964037    2573 factory.go:223] Registration of the crio container factory successfully
Nov 04 02:31:46.964161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964077    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 02:31:46.964161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964092    2573 factory.go:103] Registering Raw factory
Nov 04 02:31:46.964161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964101    2573 manager.go:1199] Started watching for new ooms in manager
Nov 04 02:31:46.964988 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.964971    2573 manager.go:319] Starting recovery of all containers
Nov 04 02:31:46.971123 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.971093    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 02:31:46.978154 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.978134    2573 manager.go:324] Recovery completed
Nov 04 02:31:46.980292 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:46.980271    2573 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Nov 04 02:31:46.983124 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.983112    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:46.984657 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.984635    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:46.984702 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.984678    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:46.984702 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.984692    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:46.985209 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985197    2573 cpu_manager.go:226] "Starting" policy="none"
Nov 04 02:31:46.985251 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985208    2573 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 02:31:46.985251 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985225    2573 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 02:31:46.985316 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985248    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:46.985979 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985961    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:46.986041 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.985994    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:46.986041 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.986005    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:46.987200 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.987188    2573 policy_none.go:50] "Start"
Nov 04 02:31:46.987244 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.987204    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 02:31:46.987244 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.987214    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 02:31:46.989046 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:46.989035    2573 policy_none.go:44] "Start"
Nov 04 02:31:47.034016 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.033881    2573 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 02:31:47.037847 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.037834    2573 manager.go:341] "Starting Device Plugin manager"
Nov 04 02:31:47.037903 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.037867    2573 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 02:31:47.037903 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.037877    2573 server.go:87] "Starting device plugin registration server"
Nov 04 02:31:47.038126 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.038114    2573 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 02:31:47.038166 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.038129    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 02:31:47.038245 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.038227    2573 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 02:31:47.038430 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.038416    2573 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 02:31:47.038430 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.038429    2573 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 02:31:47.038881 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.038864    2573 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 02:31:47.038956 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.038913    2573 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:31:47.104913 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.104881    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-201.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 02:31:47.104913 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.104884    2573 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-201.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 02:31:47.106915 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.104880    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-201.us-west-1.compute.internal.18e4b6c490834696  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-201.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-201.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:31:46.95466767 +0000 UTC m=+0.333551151,LastTimestamp:2026-11-04 02:31:46.95466767 +0000 UTC m=+0.333551151,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-201.us-west-1.compute.internal,}"
Nov 04 02:31:47.109399 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.109364    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 02:31:47.109482 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.109403    2573 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 02:31:47.109482 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.109420    2573 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 02:31:47.109482 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.109450    2573 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 02:31:47.138788 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.138738    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:47.139469 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.139450    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:47.139549 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.139481    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:47.139549 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.139499    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:47.139549 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.139526    2573 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.181825 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.181810    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 02:31:47.181912 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.181844    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l8kmr"
Nov 04 02:31:47.210792 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.210769    2573 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"]
Nov 04 02:31:47.210865 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.210822    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:47.211623 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.211603    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:47.211697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.211633    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:47.211697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.211643    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:47.213811 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.213799    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:47.213969 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.213954    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.214007 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.213985    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:47.214582 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214561    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:47.214674 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214597    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:47.214674 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214610    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:47.214674 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214569    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:47.214818 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214679    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:47.214818 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.214694    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:47.216777 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.216761    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.216847 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.216787    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:31:47.217476 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.217461    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:31:47.217560 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.217489    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:31:47.217560 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.217503    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:31:47.245063 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.245041    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-201.us-west-1.compute.internal\" not found" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.251512 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.251495    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-201.us-west-1.compute.internal\" not found" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.264409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.264393    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.264483 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.264417    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.264483 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.264436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.266205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.266191    2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:47.317366 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.317350    2573 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:47.365259 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.365366 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365270    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.365366 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.365366 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365321    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.365548 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365393    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.365548 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.365427    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.366324 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.366307    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:47.367357 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.367343    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:47.367532 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.367520    2573 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.367565 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.367544    2573 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-201.us-west-1.compute.internal\": node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:31:47.463667 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.463617    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.547855 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.547830    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.554665 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.554647    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.584687 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.584667    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:31:47.585317 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.585306    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:31:47.688718 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.688696    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:31:47.894960 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.894890    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 02:31:47.895417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.895014    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 02:31:47.895417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.895027    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 02:31:47.895417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.895038    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 02:31:47.895417 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:47.895091    2573 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a7878fea1d4a142dfa615cdc355a2760-4128f5fe3c6c09e5.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-128-201.us-west-1.compute.internal.18e4b6c4924d774c\": read tcp 10.0.128.201:52496->3.138.50.34:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-128-201.us-west-1.compute.internal.18e4b6c4924d774c  default   4806 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-201.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-201.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-201.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:31:46 +0000 UTC,LastTimestamp:2026-11-04 02:31:47.139505038 +0000 UTC m=+0.518388529,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-201.us-west-1.compute.internal,}"
Nov 04 02:31:47.940221 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.940201    2573 apiserver.go:52] "Watching apiserver"
Nov 04 02:31:47.962820 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:47.962801    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 02:31:48.186540 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.186463    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:26:48 +0000 UTC" deadline="2028-06-09 10:24:11.588259157 +0000 UTC"
Nov 04 02:31:48.186540 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.186494    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13999h52m23.401769608s"
Nov 04 02:31:48.203071 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.203044    2573 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 02:31:48.203394 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.203359    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zrpnk","openshift-network-diagnostics/network-check-target-jqtjk","kube-system/global-pull-secret-syncer-8kq82","kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal","openshift-dns/node-resolver-xs98j","openshift-multus/multus-c7lsk","openshift-multus/network-metrics-daemon-7t4hg","openshift-network-operator/iptables-alerter-dsbf4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal","openshift-ovn-kubernetes/ovnkube-node-2g4ht","openshift-image-registry/node-ca-jgvkx","openshift-multus/multus-additional-cni-plugins-n8nf4","kube-system/konnectivity-agent-l8xbh"]
Nov 04 02:31:48.205783 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.205766    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.207771 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.207753    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:48.207840 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.207822    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:48.209723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.209708    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.209781 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.209766    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:48.211693 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.211681    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.213687 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.213672    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.215623 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.215610    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.215701 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.215667    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:48.217606 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.217594    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.219704 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.219685    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.223884 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.222314    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.225656 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.225633    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.227880 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.227860    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.229987 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.229967    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.257846 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.257817    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.257846 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.257844    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gjd4x\""
Nov 04 02:31:48.257984 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.257866    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.263419 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.263402    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.263518 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.263441    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.263518 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.263516    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vd7gf\""
Nov 04 02:31:48.264032 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.264015    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 02:31:48.265314 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.265295    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.265407 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.265360    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 02:31:48.265497 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.265485    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.265497 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.265491    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 02:31:48.265571 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.265492    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ffw8m\""
Nov 04 02:31:48.269201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269188    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4v2qx\""
Nov 04 02:31:48.269337 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269323    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 02:31:48.269433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269402    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.269433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269418    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.269506 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.269506 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269455    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.269506 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269479    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.269605 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269513    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.269605 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269532    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.269605 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269566    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.269719 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.269719 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269638    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.269719 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269673    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.269719 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269704    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.269843 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269737    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.269843 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269756    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdn4\" (UniqueName: \"kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.269843 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269785    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:48.269843 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269825    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.269973 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269856    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.269973 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.269973 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.269973 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.269958    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270109    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270153    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270200    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270273    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270304    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270349    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.270458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270460    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270485    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270516    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270536    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270554    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270635    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270649    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270669    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.270697 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270700    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270714    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdx69\" (UniqueName: \"kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270727    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270777    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jbmx\" (UniqueName: \"kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270796    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270824    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270869    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270884    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270944    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d8pz\" (UniqueName: \"kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.270986    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271004    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.271050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271037    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271049    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271062    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271132    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgs97\" (UniqueName: \"kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271145    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgkt\" (UniqueName: \"kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271158    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271168    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271206    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271228    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271244    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271257    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271271    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2cr\" (UniqueName: \"kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271322    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6dhp\" (UniqueName: \"kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.271751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271443    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271492    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fm8qv\""
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271496    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271547    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271572    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.272201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.271636    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.358340 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.356240    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 02:31:48.358340 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.356302    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q9zgb"
Nov 04 02:31:48.358930 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.358907    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.358930 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.358925    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 02:31:48.359076 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359020    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 02:31:48.359076 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359033    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 02:31:48.359076 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359042    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 02:31:48.359076 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359051    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.359076 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359055    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359089    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359101    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359108    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359151    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359154    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.359267 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359158    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 02:31:48.359467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359271    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lg9c8\""
Nov 04 02:31:48.359467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359275    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmwv6\""
Nov 04 02:31:48.359847 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359832    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 02:31:48.359920 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359905    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6l7l7\""
Nov 04 02:31:48.359977 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.359965    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rgzgm\""
Nov 04 02:31:48.372178 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372271 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372190    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372271 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372214    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.372271 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372238    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372299    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372304    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372304    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372329    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372409    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372431    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372443    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372476    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372481    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372500    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372497    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372513    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372542    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372542    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372544    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372555    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdn4\" (UniqueName: \"kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372734    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.372803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372793    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372828    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372857    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372904    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372927    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372959    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372959    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372973    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372997    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373001    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.372999    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373008    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373008    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373041    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373041    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.373437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373081    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373114    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373145    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373190    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373215    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373265    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373294    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373310    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373334    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373343    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373386    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373403    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373411    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.374203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373462    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373505    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373505    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373515    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373527    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373540    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373565    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373540    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373578    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373568    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373599    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373617    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373618    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373655    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373661    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373661    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373245    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375031 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373701    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373709    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373701    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373735    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373735    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373770    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373774    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdx69\" (UniqueName: \"kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373791    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373814    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jbmx\" (UniqueName: \"kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373862    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373871    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373883    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373894    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373921    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373951    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.373995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.375692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374016    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374068    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374082    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8pz\" (UniqueName: \"kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374097    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374121    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374168    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374174    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374179    2573 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374227    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374252    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374274    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374298    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374322    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374388    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374406    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374413    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgs97\" (UniqueName: \"kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374442    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374322    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374468    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374507    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374515    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374387    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374612    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgkt\" (UniqueName: \"kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374638    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374608    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374685    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374712    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374734    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374757    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2cr\" (UniqueName: \"kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374780    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6dhp\" (UniqueName: \"kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.376680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374802    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374825    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374836    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374847    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374880    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374898    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374903    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.374916    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374933    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374946    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.374992    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.374996    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:48.874959241 +0000 UTC m=+2.253842727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375030    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375032    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375077    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375100    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375173    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377115 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.375260    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.375307    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:48.875291052 +0000 UTC m=+2.254174551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375356    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375655    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.375661    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.377501    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.377559    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.377642 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.377625    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.377830 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.377769    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.431453 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.431430    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:31:48.431453 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.431447    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:31:48.431453 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.431456    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:48.431655 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:48.431535    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:48.931518501 +0000 UTC m=+2.310401990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:48.434182 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.434141    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdn4\" (UniqueName: \"kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:48.434561 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.434535    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.434904 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.434881    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdx69\" (UniqueName: \"kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.435363 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.435339    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8pz\" (UniqueName: \"kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.435569 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.435548    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jbmx\" (UniqueName: \"kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:48.436060 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.436036    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgkt\" (UniqueName: \"kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.436417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.436392    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2cr\" (UniqueName: \"kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.436608 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.436563    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6dhp\" (UniqueName: \"kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.436660 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.436605    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgs97\" (UniqueName: \"kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.535677 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.535652    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:31:48.561653 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.561630    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:31:48.563758 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.563739    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:31:48.581562 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.581532    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7lsk"
Nov 04 02:31:48.595279 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.595250    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:31:48.625495 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.625469    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:31:48.627578 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.627556    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:31:48.661115 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.661094    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:48.667756 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:48.667736    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:31:49.840819 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:49.840758    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41ac5ecba6dc9accfa45177979cc284.slice/crio-d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4 WatchSource:0}: Error finding container d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4: Status 404 returned error can't find the container with id d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4
Nov 04 02:31:49.841455 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:49.841432    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3d33261584ab125395f2a23dbce857.slice/crio-1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c WatchSource:0}: Error finding container 1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c: Status 404 returned error can't find the container with id 1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c
Nov 04 02:31:49.841886 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:49.841848    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634bdde3_359f_4989_8c42_12f216ccfbc8.slice/crio-4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b WatchSource:0}: Error finding container 4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b: Status 404 returned error can't find the container with id 4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b
Nov 04 02:31:49.842518 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:49.842498    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef07bfa_aa26_4365_b160_67835f20cbd5.slice/crio-bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156 WatchSource:0}: Error finding container bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156: Status 404 returned error can't find the container with id bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156
Nov 04 02:31:49.846404 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:49.846387    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:31:49.961732 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:49.961702    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:50.028391 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.028364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:50.028503 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.028398    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:50.028503 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.028493    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:50.028610 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.028520    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:50.028610 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.028536    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:51.028524407 +0000 UTC m=+3.259707784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:50.028610 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.028576    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:51.028559649 +0000 UTC m=+3.259743025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:50.129524 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.129459    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:50.130112 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.129618    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:31:50.130112 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.129634    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:31:50.130112 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.129642    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:50.130112 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:50.129701    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:51.129690098 +0000 UTC m=+3.360873485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:50.261347 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.261290    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerStarted","Data":"4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b"}
Nov 04 02:31:50.262334 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.262307    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"b41ac5ecba6dc9accfa45177979cc284","Type":"ContainerStarted","Data":"d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4"}
Nov 04 02:31:50.263728 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.263705    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8xbh" event={"ID":"def07bfa-aa26-4365-b160-67835f20cbd5","Type":"ContainerStarted","Data":"bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156"}
Nov 04 02:31:50.264727 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.264704    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerStarted","Data":"1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c"}
Nov 04 02:31:50.425958 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.425879    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:50.437500 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.437480    2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:31:50.489809 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.489783    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ba6327_ae4e_46ce_b357_9787f3c1e807.slice/crio-6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75 WatchSource:0}: Error finding container 6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75: Status 404 returned error can't find the container with id 6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75
Nov 04 02:31:50.507064 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.507021    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:26:49 +0000 UTC" deadline="2028-06-30 13:27:07.269413784 +0000 UTC"
Nov 04 02:31:50.507064 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:50.507061    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14506h55m16.76235673s"
Nov 04 02:31:50.554051 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.554022    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ef75a9_a7c5_4c65_9515_0e25b697cfd4.slice/crio-7b32e8d0f109d00907843cb7f6675698066a9edc89c7382407a18d9ab34b9630 WatchSource:0}: Error finding container 7b32e8d0f109d00907843cb7f6675698066a9edc89c7382407a18d9ab34b9630: Status 404 returned error can't find the container with id 7b32e8d0f109d00907843cb7f6675698066a9edc89c7382407a18d9ab34b9630
Nov 04 02:31:50.594064 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.594033    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa40eff_cd47_4cf7_98f9_f6ceb6d511ea.slice/crio-d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53 WatchSource:0}: Error finding container d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53: Status 404 returned error can't find the container with id d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53
Nov 04 02:31:50.612818 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.612795    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fd8012_9c9e_409e_86c1_527df4c73731.slice/crio-bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6 WatchSource:0}: Error finding container bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6: Status 404 returned error can't find the container with id bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6
Nov 04 02:31:50.631997 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.631976    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f18f1cf_0fd4_4c98_8550_ed2539a16bfe.slice/crio-92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31 WatchSource:0}: Error finding container 92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31: Status 404 returned error can't find the container with id 92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31
Nov 04 02:31:50.650528 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.650506    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cd0132_b993_48e2_8d27_e4a0145bf29d.slice/crio-f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029 WatchSource:0}: Error finding container f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029: Status 404 returned error can't find the container with id f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029
Nov 04 02:31:50.683836 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:31:50.683814    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b17382_cc28_4fed_8626_5ebb3d863a98.slice/crio-c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a WatchSource:0}: Error finding container c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a: Status 404 returned error can't find the container with id c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a
Nov 04 02:31:51.035100 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.035006    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:51.035100 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.035052    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:51.035476 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.035208    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:51.035476 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.035259    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:53.035243358 +0000 UTC m=+5.266426732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:51.035476 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.035306    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:51.035476 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.035336    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:53.035323697 +0000 UTC m=+5.266507078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:51.136246 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.136209    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:51.136755 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.136365    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:31:51.136755 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.136384    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:31:51.136755 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.136396    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:51.136755 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.136450    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:53.136432441 +0000 UTC m=+5.367615803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:51.257536 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.257340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:51.257536 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.257386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:51.257536 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.257340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:51.257536 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.257484    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:51.257848 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.257575    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:51.257848 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:51.257652    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:51.270410 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.269206    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a"}
Nov 04 02:31:51.270612 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.270590    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7lsk" event={"ID":"d3cd0132-b993-48e2-8d27-e4a0145bf29d","Type":"ContainerStarted","Data":"f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029"}
Nov 04 02:31:51.272792 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.272767    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgvkx" event={"ID":"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe","Type":"ContainerStarted","Data":"92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31"}
Nov 04 02:31:51.279249 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.279224    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zrpnk" event={"ID":"41ef75a9-a7c5-4c65-9515-0e25b697cfd4","Type":"ContainerStarted","Data":"7b32e8d0f109d00907843cb7f6675698066a9edc89c7382407a18d9ab34b9630"}
Nov 04 02:31:51.281161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.281135    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xs98j" event={"ID":"a2ba6327-ae4e-46ce-b357-9787f3c1e807","Type":"ContainerStarted","Data":"6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75"}
Nov 04 02:31:51.283195 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.283175    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dsbf4" event={"ID":"33fd8012-9c9e-409e-86c1-527df4c73731","Type":"ContainerStarted","Data":"bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6"}
Nov 04 02:31:51.284556 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.284509    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8" event={"ID":"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea","Type":"ContainerStarted","Data":"d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53"}
Nov 04 02:31:51.507944 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.507836    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:26:49 +0000 UTC" deadline="2028-08-04 06:34:09.097076824 +0000 UTC"
Nov 04 02:31:51.507944 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:51.507871    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15340h2m17.589209658s"
Nov 04 02:31:53.050701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.050499    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:53.050701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.050674    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:53.051233 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.050787    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:53.051233 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.050833    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:57.050820603 +0000 UTC m=+9.282003976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:53.051233 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.050882    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:53.051233 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.050910    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:57.050899091 +0000 UTC m=+9.282082469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:53.151347 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.151313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:53.151496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.151454    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:31:53.151496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.151467    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:31:53.151496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.151475    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:53.151613 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.151513    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:31:57.151502105 +0000 UTC m=+9.382685467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:53.257944 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.257836    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:53.258109 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.257979    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:53.258109 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.258057    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:53.258227 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.258145    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:53.258227 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.258199    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:53.258333 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:53.258259    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:53.298922 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.298883    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="e57c07b2edd7801a79ac11e364d0fdc20661caf53dbf7ee05a2930c919988967" exitCode=0
Nov 04 02:31:53.299105 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:53.298942    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"e57c07b2edd7801a79ac11e364d0fdc20661caf53dbf7ee05a2930c919988967"}
Nov 04 02:31:54.305952 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.305489    2573 generic.go:356] "Generic (PLEG): container finished" podID="1e3d33261584ab125395f2a23dbce857" containerID="cc1ae284d486f2e80e0c696d6004833f192d9b484aa53d3cca8a5fba82fd1b4e" exitCode=0
Nov 04 02:31:54.305952 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.305591    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerDied","Data":"cc1ae284d486f2e80e0c696d6004833f192d9b484aa53d3cca8a5fba82fd1b4e"}
Nov 04 02:31:54.309207 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.308728    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8xbh" event={"ID":"def07bfa-aa26-4365-b160-67835f20cbd5","Type":"ContainerStarted","Data":"d5f28920c1d76d7919d064e698d53faf61de09c4eec7ab981d6d05873a2caaf2"}
Nov 04 02:31:54.313050 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.312154    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"b41ac5ecba6dc9accfa45177979cc284","Type":"ContainerStarted","Data":"23c05323942e2240bb903058c67007a7c16ee355121a323a729d85cd643edaf4"}
Nov 04 02:31:54.648685 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.648101    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal" podStartSLOduration=6.648064444 podStartE2EDuration="6.648064444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:54.647900762 +0000 UTC m=+6.879084146" watchObservedRunningTime="2026-11-04 02:31:54.648064444 +0000 UTC m=+6.879247828"
Nov 04 02:31:54.649909 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:54.649405    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l8xbh" podStartSLOduration=2.806425391 podStartE2EDuration="6.649394387s" totalImagesPullingTime="3.842968996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:54.534851842 +0000 UTC m=+6.766035226" watchObservedRunningTime="2026-11-04 02:31:54.649394387 +0000 UTC m=+6.880577953"
Nov 04 02:31:55.257849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:55.257812    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:55.258015 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:55.257953    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:55.258152 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:55.258032    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:55.258152 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:55.258122    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:55.258276 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:55.258178    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:55.258276 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:55.258253    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:55.318776 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:55.318335    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerStarted","Data":"ed5b90c73e2152adbe8fd36ef15f7f9275a62d7be6d06332306b81c429933fa6"}
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.084640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.084696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.084852    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.084914    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:05.084897478 +0000 UTC m=+17.316080857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.084975    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:57.085105 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.084997    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:05.084990273 +0000 UTC m=+17.316173647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:31:57.185901 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.185585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:57.185901 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.185752    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:31:57.185901 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.185778    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:31:57.185901 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.185793    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:57.185901 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.185851    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:05.185833545 +0000 UTC m=+17.417016925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:31:57.257950 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.257915    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:57.258169 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.258092    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:57.258169 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.258163    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:57.258282 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.258241    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:57.258336 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.258287    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:57.258386 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:57.258350    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:57.327886 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:57.327830    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8" event={"ID":"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea","Type":"ContainerStarted","Data":"dcc14a5e428f9704f3d9f82ee3e790eeaa551baa148e2a0e16689d293e431b87"}
Nov 04 02:31:58.261832 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.261482    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:58.261832 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.261583    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:31:58.333942 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.333912    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgvkx" event={"ID":"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe","Type":"ContainerStarted","Data":"f813a613a19e5905fa2644e8817fdd53551536da573ee52e95a60ecb7e1a19c9"}
Nov 04 02:31:58.377671 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.376868    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" podStartSLOduration=10.376849317 podStartE2EDuration="10.376849317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:55.434905001 +0000 UTC m=+7.666088384" watchObservedRunningTime="2026-11-04 02:31:58.376849317 +0000 UTC m=+10.608032701"
Nov 04 02:31:58.491060 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.491006    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jgvkx" podStartSLOduration=3.561600466 podStartE2EDuration="10.490985328s" totalImagesPullingTime="6.929384862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:58.490343212 +0000 UTC m=+10.721526595" watchObservedRunningTime="2026-11-04 02:31:58.490985328 +0000 UTC m=+10.722168712"
Nov 04 02:31:58.839288 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:58.839239    2573 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 02:31:59.191564 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.191418    2573 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:31:58.839259789Z","UUID":"3fb76a9f-3a44-4c32-bcf3-62eb01ce2e5a","Handler":null,"Name":"","Endpoint":""}
Nov 04 02:31:59.193360 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.193163    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 02:31:59.193479 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.193368    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 02:31:59.257755 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.257730    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:31:59.257865 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.257780    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:31:59.257925 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.257887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:31:59.257925 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:59.257899    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:31:59.258032 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:59.258010    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:31:59.258131 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:31:59.258114    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:31:59.337733 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.337697    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xs98j" event={"ID":"a2ba6327-ae4e-46ce-b357-9787f3c1e807","Type":"ContainerStarted","Data":"365094ab3e9f776b17f81eb6da91bbb8b0910df19b087d0365f8b348abce5316"}
Nov 04 02:31:59.340036 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.340006    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="72475cb38757846b5e0346b2527cf583906d7e8218315f1debb05637649d1800" exitCode=0
Nov 04 02:31:59.340182 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.340065    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"72475cb38757846b5e0346b2527cf583906d7e8218315f1debb05637649d1800"}
Nov 04 02:31:59.342936 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.342869    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8" event={"ID":"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea","Type":"ContainerStarted","Data":"69f40637bb4a9d4fe06379d20ad9f145be518bbd94202221466c9230d2430e30"}
Nov 04 02:31:59.345281 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.345258    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7lsk" event={"ID":"d3cd0132-b993-48e2-8d27-e4a0145bf29d","Type":"ContainerStarted","Data":"9bf5b17242ab28600adb3ecfc6d2547d21d128a7e096b5ec1ab3b60a7c606484"}
Nov 04 02:31:59.452190 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.452100    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-xs98j" podStartSLOduration=3.4206854509999998 podStartE2EDuration="11.452060179s" totalImagesPullingTime="8.031374728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:59.451801553 +0000 UTC m=+11.682984935" watchObservedRunningTime="2026-11-04 02:31:59.452060179 +0000 UTC m=+11.683243562"
Nov 04 02:31:59.689851 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:31:59.689791    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-c7lsk" podStartSLOduration=4.066985114 podStartE2EDuration="11.689771941s" totalImagesPullingTime="7.622786827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:59.689441124 +0000 UTC m=+11.920624507" watchObservedRunningTime="2026-11-04 02:31:59.689771941 +0000 UTC m=+11.920955326"
Nov 04 02:32:01.258090 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.258002    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:01.258090 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.258034    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:01.258614 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.258129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:01.258614 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:01.258134    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:01.258614 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:01.258199    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:01.258614 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:01.258249    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:01.350025 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.349992    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zrpnk" event={"ID":"41ef75a9-a7c5-4c65-9515-0e25b697cfd4","Type":"ContainerStarted","Data":"b83e0ba09e9fd964b647c7d8c8d9bc1de9295d5bda9f9feec728027bcfde35dd"}
Nov 04 02:32:01.352011 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.351989    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="8632650a710ff7ef0e88a47c8f24ea0d4aea0a80105353cba0f8b621c8fc930f" exitCode=0
Nov 04 02:32:01.352112 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.352056    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"8632650a710ff7ef0e88a47c8f24ea0d4aea0a80105353cba0f8b621c8fc930f"}
Nov 04 02:32:01.353460 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.353427    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dsbf4" event={"ID":"33fd8012-9c9e-409e-86c1-527df4c73731","Type":"ContainerStarted","Data":"4192d910d45397eebd083124ddc1bb46076ca06ae606b16acb49ece45c631f1a"}
Nov 04 02:32:01.355550 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.355526    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8" event={"ID":"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea","Type":"ContainerStarted","Data":"8cede57aebdae011307dd31d43530957937903c68062a9eca0fe7655b2405464"}
Nov 04 02:32:01.592480 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.592392    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zrpnk" podStartSLOduration=3.591213334 podStartE2EDuration="13.592379361s" totalImagesPullingTime="10.001166027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:32:01.467797747 +0000 UTC m=+13.698981130" watchObservedRunningTime="2026-11-04 02:32:01.592379361 +0000 UTC m=+13.823562743"
Nov 04 02:32:01.705619 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.705567    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dsbf4" podStartSLOduration=5.115521984 podStartE2EDuration="13.705554818s" totalImagesPullingTime="8.590032834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:32:01.705200756 +0000 UTC m=+13.936384139" watchObservedRunningTime="2026-11-04 02:32:01.705554818 +0000 UTC m=+13.936738200"
Nov 04 02:32:01.822616 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:01.822561    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8" podStartSLOduration=3.839739198 podStartE2EDuration="13.822548781s" totalImagesPullingTime="9.982809583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:32:01.822289404 +0000 UTC m=+14.053472799" watchObservedRunningTime="2026-11-04 02:32:01.822548781 +0000 UTC m=+14.053732164"
Nov 04 02:32:03.257553 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:03.257522    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:03.258253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:03.257578    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:03.258253 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:03.257688    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:03.258253 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:03.257705    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:03.258253 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:03.257799    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:03.258253 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:03.257888    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:04.363896 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:04.363711    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="dca946dd3b8b5a6b267ce078b628a2a7c2f035b66bd88999a2e1689f95165a8c" exitCode=0
Nov 04 02:32:04.363896 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:04.363797    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"dca946dd3b8b5a6b267ce078b628a2a7c2f035b66bd88999a2e1689f95165a8c"}
Nov 04 02:32:05.147492 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.147457    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:05.147636 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.147531    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:05.147636 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.147607    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:05.147636 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.147607    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:05.147738 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.147673    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:21.147655249 +0000 UTC m=+33.378838628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:05.147738 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.147687    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:21.147681097 +0000 UTC m=+33.378864458 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:05.248597 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.248566    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:05.248739 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.248679    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:32:05.248739 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.248692    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:32:05.248739 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.248702    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:05.248834 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.248750    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:21.24873456 +0000 UTC m=+33.479917951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:05.257741 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.257723    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:05.257849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.257757    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:05.257931 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.257903    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:05.257987 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.257948    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:05.258063 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.258047    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:05.258165 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:05.258148    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:05.952407 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.952370    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:32:05.952866 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.952547    2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 02:32:05.953254 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:05.953231    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:32:07.257358 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:07.257333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:07.257358 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:07.257349    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:07.257737 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:07.257367    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:07.257737 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:07.257489    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:07.257737 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:07.257561    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:07.257737 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:07.257628    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:07.373389 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:07.373299    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="3927af249fd33b644cef7ab4e4d08956f90816cf027666dc7defb4f3700df365" exitCode=0
Nov 04 02:32:07.373389 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:07.373350    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"3927af249fd33b644cef7ab4e4d08956f90816cf027666dc7defb4f3700df365"}
Nov 04 02:32:08.377927 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:08.377892    2573 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="7bc9f86c131521db25f10f0c2af6045c6db514358097f08404e6f7b05a947b62" exitCode=0
Nov 04 02:32:08.378311 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:08.377939    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerDied","Data":"7bc9f86c131521db25f10f0c2af6045c6db514358097f08404e6f7b05a947b62"}
Nov 04 02:32:09.258352 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:09.258323    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:09.258500 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:09.258361    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:09.258500 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:09.258333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:09.258500 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:09.258450    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:09.258611 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:09.258508    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:09.258611 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:09.258565    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:09.382403 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:09.382374    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" event={"ID":"634bdde3-359f-4989-8c42-12f216ccfbc8","Type":"ContainerStarted","Data":"d573c9de8b7f7b9b86aec1e0da829e2545c5c0461d13560ca0055a9095a86406"}
Nov 04 02:32:09.553454 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:09.553356    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n8nf4" podStartSLOduration=7.720330928 podStartE2EDuration="21.5533422s" totalImagesPullingTime="13.833011272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:32:09.553138993 +0000 UTC m=+21.784322376" watchObservedRunningTime="2026-11-04 02:32:09.5533422 +0000 UTC m=+21.784525582"
Nov 04 02:32:11.258197 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:11.258167    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:11.258197 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:11.258178    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:11.258197 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:11.258191    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:11.258816 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:11.258273    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:11.258816 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:11.258372    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:11.258816 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:11.258457    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:13.258364 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:13.258327    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:13.258818 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:13.258382    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:13.258818 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:13.258478    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:13.258818 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:13.258484    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:13.258818 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:13.258626    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:13.258818 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:13.258712    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:15.258345 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:15.258312    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:15.258801 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:15.258313    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:15.258801 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:15.258426    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:15.258801 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:15.258418    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:15.258801 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:15.258508    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:15.258801 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:15.258578    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:17.258017 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:17.257981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:17.258463 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:17.258047    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:17.258463 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:17.258137    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:17.258463 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:17.258147    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:17.258463 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:17.258198    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:17.258463 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:17.258292    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:19.257511 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:19.257475    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:19.257511 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:19.257499    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:19.257972 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:19.257518    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:19.257972 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:19.257611    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:19.257972 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:19.257726    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:19.257972 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:19.257799    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:21.161171 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.161140    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:21.161638 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.161178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:21.161638 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.161289    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:21.161638 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.161303    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:21.161638 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.161335    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:53.161322438 +0000 UTC m=+65.392505812 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:21.161638 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.161366    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:53.161347692 +0000 UTC m=+65.392531072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:21.257541 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.257512    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:21.257541 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.257537    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:21.257723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.257608    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:21.257723 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.257623    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:21.257723 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.257683    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:21.257813 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.257746    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:21.261607 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:21.261593    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:21.261705 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.261694    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:32:21.261743 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.261710    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:32:21.261743 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.261718    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:21.261810 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:21.261753    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:32:53.261740441 +0000 UTC m=+65.492923803 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:22.587167 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:22.587132    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:22.785918 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:22.785876    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:23.258118 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:23.258090    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:23.258269 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:23.258092    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:23.258269 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:23.258188    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:23.258269 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:23.258090    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:23.258269 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:23.258261    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:23.258395 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:23.258316    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:25.257483 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:25.257439    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:25.257483 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:25.257485    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:25.257960 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:25.257570    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:25.257960 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:25.257575    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:25.257960 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:25.257645    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:25.257960 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:25.257716    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:25.951365 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:25.951326    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:26.090997 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:26.090971    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:26.443256 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:26.443223    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:26.590709 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:26.590671    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:27.258157 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:27.258124    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:27.258157 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:27.258157    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:27.258360 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:27.258125    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:27.258360 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:27.258242    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:27.258360 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:27.258283    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:27.258360 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:27.258351    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:29.257754 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:29.257728    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:29.258163 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:29.257832    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:29.258163 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:29.257916    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:29.258163 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:29.257993    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:29.258163 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:29.257996    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:29.258163 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:29.258055    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:30.291278 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:30.291242    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:30.431137 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:30.431109    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:31.258310 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:31.258272    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:31.258488 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:31.258342    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:31.258488 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:31.258455    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:31.258610 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:31.258500    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:31.258610 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:31.258585    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:31.258709 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:31.258658    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:31.465814 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:31.465779    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:32.766899 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:32.766860    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:32.907530 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:32.907497    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:33.257667 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:33.257640    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:33.257667 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:33.257664    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:33.257850 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:33.257757    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:33.257850 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:33.257795    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:33.257930 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:33.257852    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:33.257964 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:33.257935    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:33.601511 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:33.601441    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:33.744582 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:33.744545    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:35.258014 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:35.257977    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:35.258433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:35.258093    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:35.258433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:35.258121    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:35.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:35.258096    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:35.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:35.258199    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:35.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:35.258271    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:37.257905 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:37.257871    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:37.258418 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:37.257871    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:37.258418 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:37.257976    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:37.258418 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:37.257871    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:37.258418 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:37.258153    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:37.258418 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:37.258183    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:39.257488 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:39.257456    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:39.257488 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:39.257486    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:39.257908 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:39.257560    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:39.257908 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:39.257600    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:39.257908 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:39.257677    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:39.257908 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:39.257727    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:41.258158 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:41.258128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:41.258625 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:41.258128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:41.258625 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:41.258245    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:41.258625 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:41.258128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:41.258625 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:41.258312    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:41.258625 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:41.258387    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:43.257626 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:43.257593    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:43.258165 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:43.257633    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:43.258165 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:43.257636    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:43.258165 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:43.257718    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:43.258165 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:43.257778    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:43.258165 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:43.257875    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:44.454042 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:44.454010    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:32:44.454469 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:44.454054    2573 generic.go:356] "Generic (PLEG): container finished" podID="d3cd0132-b993-48e2-8d27-e4a0145bf29d" containerID="9bf5b17242ab28600adb3ecfc6d2547d21d128a7e096b5ec1ab3b60a7c606484" exitCode=1
Nov 04 02:32:44.454469 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:44.454126    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7lsk" event={"ID":"d3cd0132-b993-48e2-8d27-e4a0145bf29d","Type":"ContainerDied","Data":"9bf5b17242ab28600adb3ecfc6d2547d21d128a7e096b5ec1ab3b60a7c606484"}
Nov 04 02:32:44.454561 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:44.454544    2573 scope.go:122] "RemoveContainer" containerID="9bf5b17242ab28600adb3ecfc6d2547d21d128a7e096b5ec1ab3b60a7c606484"
Nov 04 02:32:45.258234 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.258198    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:45.258234 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.258217    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:45.258449 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:45.258321    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:45.258449 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.258370    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:45.258449 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:45.258440    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:45.258542 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:45.258502    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:45.458251 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.458227    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:32:45.458690 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.458293    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7lsk" event={"ID":"d3cd0132-b993-48e2-8d27-e4a0145bf29d","Type":"ContainerStarted","Data":"b033f126c32aacbe1b1dcfade7a340bc22012224bcc42902e5447b7ac87362a1"}
Nov 04 02:32:45.883130 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:45.883096    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:46.025795 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:46.025756    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:46.167881 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:46.167797    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:46.369599 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:46.369571    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:46.515667 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:46.515626    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:47.190862 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.190721    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:47.257998 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.257975    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:47.258108 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.258084    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:47.258108 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.258093    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:47.258190 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:47.258092    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:47.258190 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:47.258158    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:47.258250 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:47.258205    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:47.388654 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.388625    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:47.465831 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.465796    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"63e6cef9bb2577e40c2a857c3c42a52e03e31cc0cfd09d4a958f7395fe4084f3"}
Nov 04 02:32:47.465966 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.465917    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"9383c28e7d0fb2a0bd0712f1cde02b841d1abb26a45c54c680a0208802011558"}
Nov 04 02:32:47.465966 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.465948    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"5da3ef14a76d642450009266354504b5b6b12ba2a523f98feddb05538c2b8a60"}
Nov 04 02:32:47.466033 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.465975    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"afdc17211fb2be1ce5abdb280aeb98e551bd91b0d7c76ddd67a841d55f4089d8"}
Nov 04 02:32:47.466033 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.466002    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"4858501c49ad3d623ad6ee0eb97256de89da61d7b3ee941dc851fca820d29919"}
Nov 04 02:32:47.466033 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.466029    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"7a0c02b927ea20e48f2aa9ca80d1ee66a6ea6d47c3b6ba5f5ceeff35f61beda6"}
Nov 04 02:32:47.530328 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.530298    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:47.877278 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:47.877252    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xs98j_a2ba6327-ae4e-46ce-b357-9787f3c1e807/dns-node-resolver/0.log"
Nov 04 02:32:48.676717 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:48.676694    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgvkx_5f18f1cf-0fd4-4c98-8550-ed2539a16bfe/node-ca/0.log"
Nov 04 02:32:48.751105 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:48.751060    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:48.902020 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:48.901979    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:49.148293 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.148262    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:49.258020 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.257996    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:49.258020 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.258012    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:49.258204 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:49.258109    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:49.258204 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.258131    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:49.258285 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:49.258210    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:49.258285 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:49.258267    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:49.290257 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.290221    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:49.434052 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.433889    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:49.473132 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:49.473094    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"9e2aca3baa6b9ba510438b768876ed8c94b56960ce24376c7c587bc60ae7ab77"}
Nov 04 02:32:51.257914 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:51.257881    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:51.257914 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:51.257906    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:51.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:51.257995    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:51.258433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:51.258021    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:51.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:51.258104    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:51.258433 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:51.258167    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:51.648333 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:51.648296    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:52.482973 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:52.482933    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" event={"ID":"76b17382-cc28-4fed-8626-5ebb3d863a98","Type":"ContainerStarted","Data":"47cebc3946b3e4a9307e254159c619d909d7cc09121a24dd5b48f45d3c52d653"}
Nov 04 02:32:52.483458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:52.483160    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:32:52.483458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:52.483214    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:32:52.499212 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:52.499189    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:32:52.759981 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:52.759930    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht" podStartSLOduration=8.351851225 podStartE2EDuration="1m4.759917965s" totalImagesPullingTime="56.40806674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:32:52.759455873 +0000 UTC m=+64.990639267" watchObservedRunningTime="2026-11-04 02:32:52.759917965 +0000 UTC m=+64.991101367"
Nov 04 02:32:53.190801 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.190722    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:53.190801 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.190766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:53.190970 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.190866    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:53.190970 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.190867    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:53.190970 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.190920    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs podName:7a83d912-788d-463f-a790-43d27a3f5765 nodeName:}" failed. No retries permitted until 2026-11-04 02:33:57.190903912 +0000 UTC m=+129.422087290 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs") pod "network-metrics-daemon-7t4hg" (UID: "7a83d912-788d-463f-a790-43d27a3f5765") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:32:53.190970 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.190933    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret podName:43a315e6-74d9-46f6-afad-3f9f3d4f98e0 nodeName:}" failed. No retries permitted until 2026-11-04 02:33:57.190927084 +0000 UTC m=+129.422110446 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret") pod "global-pull-secret-syncer-8kq82" (UID: "43a315e6-74d9-46f6-afad-3f9f3d4f98e0") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:32:53.258305 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.258266    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:53.258305 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.258294    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:53.258305 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.258313    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:53.258534 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.258383    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:53.258534 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.258432    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:53.258534 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.258524    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:53.291678 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.291658    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:53.291792 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.291780    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:32:53.291824 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.291796    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:32:53.291824 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.291805    2573 projected.go:196] Error preparing data for projected volume kube-api-access-w8mjs for pod openshift-network-diagnostics/network-check-target-jqtjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:53.291879 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:53.291841    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs podName:56433f74-7032-4c74-ab37-3f80599894a4 nodeName:}" failed. No retries permitted until 2026-11-04 02:33:57.291830643 +0000 UTC m=+129.523014017 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w8mjs" (UniqueName: "kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs") pod "network-check-target-jqtjk" (UID: "56433f74-7032-4c74-ab37-3f80599894a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:32:53.484999 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.484922    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:32:53.500422 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.500399    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:32:53.976739 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:53.976703    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:54.185621 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:54.185582    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:54.394593 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:54.394563    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:54.537463 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:54.537438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:54.882862 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:54.882831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:55.257541 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:55.257523    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:55.257701 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:55.257616    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:55.257701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:55.257527    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:55.257701 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:55.257674    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:55.257701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:55.257527    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:55.257917 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:55.257726    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:55.481724 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:55.481693    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:56.085442 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:56.085405    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:56.280613 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:56.280583    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:56.678964 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:56.678932    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:56.878599 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:56.878565    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:57.257407 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:57.257377    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:57.257407 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:57.257413    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:57.257911 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:57.257483    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:57.257911 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:57.257505    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:57.257911 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:57.257551    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:57.257911 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:57.257647    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:57.481427 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:57.481398    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:58.081225 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:58.081185    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:58.282882 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:58.282841    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:58.678438 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:58.678399    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:58.881180 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:58.881149    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:32:59.257866 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:59.257830    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:32:59.258053 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:59.257930    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:32:59.258053 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:59.257941    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:32:59.258053 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:59.258027    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:32:59.258262 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:59.258029    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:32:59.258262 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:32:59.258146    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:32:59.478304 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:32:59.478273    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:00.083195 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:00.083158    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:00.227539 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:00.227503    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:00.372823 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:00.372743    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:00.683099 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:00.683005    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:00.880049 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:00.880020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:01.257515 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:01.257481    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:01.257515 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:01.257511    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:01.257767 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:01.257599    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:01.257767 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:01.257648    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:01.257767 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:01.257743    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:01.257891 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:01.257769    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:01.488026 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:01.487985    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:01.629679 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:01.629602    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:02.082269 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:02.082221    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:02.291821 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:02.291782    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:02.683833 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:02.683799    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:02.899556 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:02.899521    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:03.054701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.054665    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:03.257774 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.257750    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:03.258189 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.257751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:03.258189 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:03.257845    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:03.258189 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.257751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:03.258189 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:03.257923    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:03.258189 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:03.257977    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:03.501206 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.501175    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:03.643489 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:03.643456    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:04.082571 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:04.082539    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:04.280142 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:04.280107    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:04.482359 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:04.482268    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:04.679686 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:04.679653    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:05.258317 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:05.258131    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:05.258502 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:05.258168    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:05.258502 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:05.258385    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:05.258502 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:05.258491    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:05.258622 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:05.258171    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:05.258622 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:05.258585    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:07.257806 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:07.257778    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:07.258183 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:07.257789    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:07.258183 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:07.257887    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:07.258183 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:07.257977    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:07.258183 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:07.257790    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:07.258183 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:07.258100    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:09.258029 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.257997    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:09.258029 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.258028    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:09.258496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:09.258110    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:09.258496 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.258155    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:09.258496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:09.258223    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:09.258496 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:09.258295    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:09.350175 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.350145    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:09.492388 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.492273    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:09.768023 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.767988    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:09.953990 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:09.953964    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:10.104449 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:10.104360    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:10.245776 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:10.245741    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:11.112022 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.111987    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:11.258387 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.258356    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:11.258576 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.258395    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:11.258641 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.258575    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:11.258641 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.258605    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:11.258836 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.258724    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:11.259354 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.259324    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:11.260378 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.260356    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kq82"]
Nov 04 02:33:11.262539 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.261235    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqtjk"]
Nov 04 02:33:11.262539 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.262495    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7t4hg"]
Nov 04 02:33:11.520940 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.520912    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:11.520940 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.520938    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:11.521251 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.521021    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:11.521251 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.521048    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:11.521251 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.521129    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:11.521251 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:11.521188    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:11.675007 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.674978    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:11.819399 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.819305    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:11.960198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:11.960169    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.161591 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.161503    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.306489 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.306462    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.480801 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.480721    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.625582 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.625538    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.767649 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.767612    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:12.953455 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:12.953426    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:13.257891 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:13.257862    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:13.257891 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:13.257875    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:13.257891 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:13.257862    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:13.258469 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:13.258046    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:13.258469 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:13.258130    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:13.258469 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:13.258197    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:15.257866 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:15.257832    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:15.257866 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:15.257855    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:15.258423 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:15.257939    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:15.258423 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:15.257957    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqtjk" podUID="56433f74-7032-4c74-ab37-3f80599894a4"
Nov 04 02:33:15.258423 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:15.258022    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq82" podUID="43a315e6-74d9-46f6-afad-3f9f3d4f98e0"
Nov 04 02:33:15.258423 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:15.258101    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7t4hg" podUID="7a83d912-788d-463f-a790-43d27a3f5765"
Nov 04 02:33:16.557303 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.557277    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeReady"
Nov 04 02:33:16.557721 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.557403    2573 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 02:33:16.712225 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.712193    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-djrmx"]
Nov 04 02:33:16.714011 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.713996    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r8bmk"]
Nov 04 02:33:16.714143 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.714127    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.715532 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.715512    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.760515 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.760486    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xf8db"]
Nov 04 02:33:16.762084 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.762048    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:16.766444 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.766425    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 02:33:16.766544 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.766432    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6tfxk\""
Nov 04 02:33:16.766544 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.766475    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 02:33:16.767316 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.767295    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 02:33:16.767509 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.767493    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 02:33:16.767684 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.767609    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt2vt\""
Nov 04 02:33:16.767684 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.767661    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 02:33:16.767787 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.767668    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 02:33:16.784156 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.784139    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-djrmx"]
Nov 04 02:33:16.810016 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.809963    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8bmk"]
Nov 04 02:33:16.846249 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.846222    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf8db"]
Nov 04 02:33:16.867497 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.867479    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 02:33:16.867589 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.867503    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 02:33:16.867589 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.867561    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-l8p2p\""
Nov 04 02:33:16.867589 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.867584    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 02:33:16.872815 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.872798    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.872862 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.872824    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.872947 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.872933    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dplk2\" (UniqueName: \"kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:16.872990 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.872978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:16.873027 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873007    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.873027 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.873119 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.873119 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873054    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.873119 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873085    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbhr\" (UniqueName: \"kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.873223 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873148    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.873223 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.873203    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zzv2\" (UniqueName: \"kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974063 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974063 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974066    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974119    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbhr\" (UniqueName: \"kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zzv2\" (UniqueName: \"kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974392 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974205    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974392 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974226    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974392 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974307    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.974392 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974332    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dplk2\" (UniqueName: \"kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:16.974392 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974375    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:16.974629 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974431    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974629 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974520    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974833 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974807    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.974945 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974862    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.975001 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.974943    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.978119 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.978096    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:16.978210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.978099    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:16.978210 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:16.978170    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:17.035259 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.035085    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zzv2\" (UniqueName: \"kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:17.035259 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.035256    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dplk2\" (UniqueName: \"kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:17.035437 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.035194    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbhr\" (UniqueName: \"kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:17.071227 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.071158    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:33:17.257990 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.257949    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:17.257990 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.257996    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:17.258248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.258018    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:17.310550 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310521    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 02:33:17.310763 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310730    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 02:33:17.310763 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310730    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-82bnw\""
Nov 04 02:33:17.311006 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310844    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8zwfm\""
Nov 04 02:33:17.311006 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310865    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9fmkz\""
Nov 04 02:33:17.311006 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310898    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 02:33:17.311006 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.310969    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 02:33:17.327049 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.326984    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:17.332621 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.332605    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:33:17.456742 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.456714    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf8db"]
Nov 04 02:33:17.537907 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.537874    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf8db" event={"ID":"db3142b7-fa88-4f52-9f8b-37beedba3df3","Type":"ContainerStarted","Data":"2d82ac74d4b1132a091ccbe4ec9c500a9f52e6cb4b6a2267d52647a49fbb2352"}
Nov 04 02:33:17.699716 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.699646    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-djrmx"]
Nov 04 02:33:17.703444 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:17.703420    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4f5551_7d57_40db_915a_fa222208f24c.slice/crio-7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8 WatchSource:0}: Error finding container 7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8: Status 404 returned error can't find the container with id 7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8
Nov 04 02:33:17.746701 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:17.746674    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8bmk"]
Nov 04 02:33:17.749966 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:17.749940    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c54c2d_0074_4af2_bbc8_5b5c734fc91b.slice/crio-4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370 WatchSource:0}: Error finding container 4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370: Status 404 returned error can't find the container with id 4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370
Nov 04 02:33:18.029292 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.029261    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:18.225059 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.225024    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:18.367430 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.367347    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:18.453091 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.453049    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"]
Nov 04 02:33:18.454945 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.454931    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.507203 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.507178    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 02:33:18.507344 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.507268    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 02:33:18.507512 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.507496    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 02:33:18.514867 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.514519    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"]
Nov 04 02:33:18.515906 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.515883    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:18.542700 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.542662    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8bmk" event={"ID":"53c54c2d-0074-4af2-bbc8-5b5c734fc91b","Type":"ContainerStarted","Data":"5b377257dffaa21de2c0c7723bf92f8d8f1cfeb155ee2f652e182e329eb1d0a1"}
Nov 04 02:33:18.542809 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.542739    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8bmk" event={"ID":"53c54c2d-0074-4af2-bbc8-5b5c734fc91b","Type":"ContainerStarted","Data":"4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370"}
Nov 04 02:33:18.543755 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.543726    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-djrmx" event={"ID":"1a4f5551-7d57-40db-915a-fa222208f24c","Type":"ContainerStarted","Data":"7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8"}
Nov 04 02:33:18.557365 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.557339    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 02:33:18.557464 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.557432    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qw9gk\""
Nov 04 02:33:18.557514 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.557477    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 02:33:18.584930 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.584897    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.585046 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.584934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.585046 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.585024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbwgq\" (UniqueName: \"kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.585198 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.585181    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.672191 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.672117    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:18.686089 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.686052    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.686170 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.686107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.686170 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.686129    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.686266 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.686168    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbwgq\" (UniqueName: \"kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.686757 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.686737    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.690082 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.689937    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.690139 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.689968    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.747851 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.747817    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbwgq\" (UniqueName: \"kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:18.765529 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:18.765500    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:33:19.133408 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:19.133373    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"]
Nov 04 02:33:19.137658 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:19.137628    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3223c5ed_97b7_46fe_baba_9e54b1b819bd.slice/crio-a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604 WatchSource:0}: Error finding container a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604: Status 404 returned error can't find the container with id a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604
Nov 04 02:33:19.548486 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:19.548444    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg" event={"ID":"3223c5ed-97b7-46fe-baba-9e54b1b819bd","Type":"ContainerStarted","Data":"a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604"}
Nov 04 02:33:20.465943 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:20.465870    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:20.552835 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:20.552793    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8bmk" event={"ID":"53c54c2d-0074-4af2-bbc8-5b5c734fc91b","Type":"ContainerStarted","Data":"8d06132985de1378be47172ad6a83fd20a2365d9f40760a6af3312dc52f0f14b"}
Nov 04 02:33:20.609790 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:20.609754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:21.374891 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.374864    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:33:21.377293 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.377262    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.483945 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.483906    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.483968    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.484026    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.484098    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.484169    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.484178    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 02:33:21.484390 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.483906    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 02:33:21.484761 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.484027    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gc8m9\""
Nov 04 02:33:21.485333 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.485310    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:33:21.508262 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508229    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508280    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508353    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508375    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508417 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508401    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.508658 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.508424    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.559705 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.559671    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-djrmx" event={"ID":"1a4f5551-7d57-40db-915a-fa222208f24c","Type":"ContainerStarted","Data":"a6227d045d4d41f7e238618eff05dbf1af01fa59eae78aef1fe9ccc8d5bad33a"}
Nov 04 02:33:21.559860 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.559721    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-djrmx" event={"ID":"1a4f5551-7d57-40db-915a-fa222208f24c","Type":"ContainerStarted","Data":"dbbd4ff19df8b46bee47e89a519c636304c282ec16040e2a702a0e976dc7614a"}
Nov 04 02:33:21.559860 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.559826    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609565    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609694    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609818    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609856    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.609897    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.610406 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.610015    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.611261 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.611047    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.611261 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.611211    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.611733 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.611662    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.614094 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.614048    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.624597 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.624574    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.671923 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.671852    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:21.678918 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.677904    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-djrmx" podStartSLOduration=2.5739961620000003 podStartE2EDuration="5.677887837s" totalImagesPullingTime="3.103891675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:21.67700169 +0000 UTC m=+93.908185075" watchObservedRunningTime="2026-11-04 02:33:21.677887837 +0000 UTC m=+93.909071224"
Nov 04 02:33:21.691019 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:21.690991    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:22.072967 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.072936    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:33:22.075124 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:22.075098    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf52ec7_7719_406b_8d32_f91721ec6c02.slice/crio-d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34 WatchSource:0}: Error finding container d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34: Status 404 returned error can't find the container with id d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34
Nov 04 02:33:22.565087 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.565031    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8bmk" event={"ID":"53c54c2d-0074-4af2-bbc8-5b5c734fc91b","Type":"ContainerStarted","Data":"3d6364e14826bfb27c8c9de3fa2ffdc625c0baa3d5f93f691d5e91e0f0c50866"}
Nov 04 02:33:22.566525 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.566501    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf8db" event={"ID":"db3142b7-fa88-4f52-9f8b-37beedba3df3","Type":"ContainerStarted","Data":"4ddf51aa161b1d2e8b814b84b8d766397f1bc0e1e1f0fbe0e581e284e0d47c4d"}
Nov 04 02:33:22.567684 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.567664    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b9446c-6hfb2" event={"ID":"fcf52ec7-7719-406b-8d32-f91721ec6c02","Type":"ContainerStarted","Data":"d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34"}
Nov 04 02:33:22.569236 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.569206    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg" event={"ID":"3223c5ed-97b7-46fe-baba-9e54b1b819bd","Type":"ContainerStarted","Data":"e75c8ef045e9c48cb783a4af03b3a7cc0dd24344a17e2bb6c85397335f6df2bc"}
Nov 04 02:33:22.569306 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.569254    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg" event={"ID":"3223c5ed-97b7-46fe-baba-9e54b1b819bd","Type":"ContainerStarted","Data":"5dbf833c3fa7871dbebc301c916018f3294a493884e10162c02f77178e3c043a"}
Nov 04 02:33:22.683436 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.683332    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r8bmk" podStartSLOduration=2.16576206 podStartE2EDuration="6.683318397s" totalImagesPullingTime="4.517556337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:33:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:22.682771074 +0000 UTC m=+94.913954459" watchObservedRunningTime="2026-11-04 02:33:22.683318397 +0000 UTC m=+94.914501779"
Nov 04 02:33:22.798996 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.798925    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xf8db" podStartSLOduration=2.043465038 podStartE2EDuration="6.798911818s" totalImagesPullingTime="4.75544678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:22.798419888 +0000 UTC m=+95.029603272" watchObservedRunningTime="2026-11-04 02:33:22.798911818 +0000 UTC m=+95.030095200"
Nov 04 02:33:22.914727 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:22.914668    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg" podStartSLOduration=2.308173632 podStartE2EDuration="4.9146532s" totalImagesPullingTime="2.606479568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:22.914188035 +0000 UTC m=+95.145371418" watchObservedRunningTime="2026-11-04 02:33:22.9146532 +0000 UTC m=+95.145836583"
Nov 04 02:33:23.886929 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:23.886898    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:24.070374 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:24.070336    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:24.436302 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:24.436266    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:25.266292 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.266244    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:25.460566 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.460531    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:25.500751 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.500722    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:33:25.685912 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.685829    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:25.865675 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.865637    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:25.886866 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.886831    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"]
Nov 04 02:33:25.891459 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.891432    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:25.943804 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.943723    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 02:33:25.943981 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.943956    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 02:33:25.944152 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.944121    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gd7d8\""
Nov 04 02:33:25.949543 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:25.948934    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"]
Nov 04 02:33:26.010547 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.010512    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:26.048712 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.048682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcszj\" (UniqueName: \"kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.048893 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.048802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.048893 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.048839    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.049013 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.048964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.100259 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.100223    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cggz4"]
Nov 04 02:33:26.104160 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.104137    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.105498 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.105452    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"]
Nov 04 02:33:26.107271 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.107250    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.149692 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.149668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcszj\" (UniqueName: \"kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.149837 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.149719    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.149837 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.149739    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.149948 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.149851    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.150788 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.150756    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.152479 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.152458    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.152479 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.152467    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.156420 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.156398    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 02:33:26.156640 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.156618    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 02:33:26.156746 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.156660    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 02:33:26.156746 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.156661    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mpgnq\""
Nov 04 02:33:26.159089 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.159055    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 02:33:26.159248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.159215    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 02:33:26.159248 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.159224    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 02:33:26.159326 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.159282    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m7pds\""
Nov 04 02:33:26.220422 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.220355    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:26.226548 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.226518    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcszj\" (UniqueName: \"kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.227707 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.227680    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"]
Nov 04 02:33:26.250605 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250580    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctfz4\" (UniqueName: \"kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.250723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.250723 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250632    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.250817 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250753    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.250817 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250790    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.250886 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250834    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.250886 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250857    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.250886 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jm6\" (UniqueName: \"kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.251019 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250890    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.251019 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.250917    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.251129 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.251110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.251169 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.251141    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.251169 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.251159    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.251261 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.251179    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.251322 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.251306    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.351802 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351773    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351811    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctfz4\" (UniqueName: \"kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351874    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351919    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351936    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.351990    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352056    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352092    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jm6\" (UniqueName: \"kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352120    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352200    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352249    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352305    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352334    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352629    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.352861 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352819    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.353201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352952    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.353201 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.352963    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.353640 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.353616    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.354796 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.354768    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.355212 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.355191    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.355564 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.355546    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.355640 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.355560    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.365287 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.365262    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:26.412467 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.412435    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jm6\" (UniqueName: \"kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.412611 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.412590    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctfz4\" (UniqueName: \"kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.421404 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.421384    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:33:26.426647 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.426625    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:33:26.430898 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:26.430877    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce3cee4_69e7_4743_ae18_ee9921c5c7d9.slice/crio-b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d WatchSource:0}: Error finding container b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d: Status 404 returned error can't find the container with id b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d
Nov 04 02:33:26.506624 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.506599    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:33:26.584133 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.584103    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b9446c-6hfb2" event={"ID":"fcf52ec7-7719-406b-8d32-f91721ec6c02","Type":"ContainerStarted","Data":"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"}
Nov 04 02:33:26.585243 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.585222    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cggz4" event={"ID":"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9","Type":"ContainerStarted","Data":"b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d"}
Nov 04 02:33:26.701372 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.701327    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7755b9446c-6hfb2" podStartSLOduration=1.515436957 podStartE2EDuration="5.701313374s" totalImagesPullingTime="4.185876417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:26.700764781 +0000 UTC m=+98.931948164" watchObservedRunningTime="2026-11-04 02:33:26.701313374 +0000 UTC m=+98.932496825"
Nov 04 02:33:26.717340 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.717293    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:26.843233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.843202    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"]
Nov 04 02:33:26.845726 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:26.845704    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7913919_20f9_4b2c_b1fe_81c2552e5c77.slice/crio-f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131 WatchSource:0}: Error finding container f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131: Status 404 returned error can't find the container with id f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131
Nov 04 02:33:26.895458 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:26.895435    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"]
Nov 04 02:33:26.897986 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:26.897960    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb675972a_9d65_4a98_9fbe_79b925302105.slice/crio-fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886 WatchSource:0}: Error finding container fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886: Status 404 returned error can't find the container with id fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886
Nov 04 02:33:27.363400 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.363366    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:27.514057 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.514022    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:27.591476 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.591388    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm" event={"ID":"f7913919-20f9-4b2c-b1fe-81c2552e5c77","Type":"ContainerStarted","Data":"f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131"}
Nov 04 02:33:27.592984 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.592959    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf" event={"ID":"b675972a-9d65-4a98-9fbe-79b925302105","Type":"ContainerStarted","Data":"098b70018d2e78b5626c09f88f1a03f72b9b8bec49f6ba292c4da36b2f3dceaa"}
Nov 04 02:33:27.593104 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.593013    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf" event={"ID":"b675972a-9d65-4a98-9fbe-79b925302105","Type":"ContainerStarted","Data":"d32645600f36245dcc5cf8b9fc1263045d42ec244ccb4fdddd27caa10b612ee9"}
Nov 04 02:33:27.593104 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.593041    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf" event={"ID":"b675972a-9d65-4a98-9fbe-79b925302105","Type":"ContainerStarted","Data":"fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886"}
Nov 04 02:33:27.662747 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:27.662719    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:28.227583 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.227546    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:28.271999 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.271967    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:33:28.274953 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.274928    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.327890 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.327861    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328203    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328232    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328257    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328308    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328364    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328364    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6g9rb\""
Nov 04 02:33:28.328409 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328384    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 02:33:28.328726 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.328419    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 02:33:28.369373 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369345    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369382    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369481    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369525    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369544    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369776    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369822    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.369892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369883    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.370312 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.370312 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.369972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.370312 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.370000    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.370312 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.370034    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.427227 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.427196    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 02:33:28.428809 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.428779    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:33:28.470471 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470611 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470513    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470611 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470538    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470611 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470574    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470611 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470600    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470624    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470661    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470742    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.470797 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470768    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.471025 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470800    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.471025 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470825    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.471025 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.470850    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.471919 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.471851    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.472032 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.471972    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.472710 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.472679    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.474353 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.474331    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.475149 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.475063    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.475301 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.475278    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.475879 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.475727    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.475879 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.475779    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.476470 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.476444    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.477146 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.477123    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.478229 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.478180    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.478395 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.478372    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.535900 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.535874    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.594884 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.594856    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:33:28.610466 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.610438    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"]
Nov 04 02:33:28.613504 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.613481    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.667205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.666616    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 02:33:28.667205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.666754    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d2bo7iqmu07ta\""
Nov 04 02:33:28.667205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.666856    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 02:33:28.667205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.666941    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 02:33:28.667205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.667005    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 02:33:28.667520 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.667324    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pf5mr\""
Nov 04 02:33:28.668064 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.667577    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 02:33:28.679322 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.679226    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"]
Nov 04 02:33:28.774899 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.774851    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775099 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.774907    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775099 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.774932    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775099 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775018    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775099 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775045    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q49df\" (UniqueName: \"kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775314 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775140    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775314 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775179    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775314 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775285    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.775460 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.775312    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876744 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876706    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876744 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876957 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876957 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876785    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876957 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49df\" (UniqueName: \"kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876957 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.876957 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876879    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.877149 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876972    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.877149 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.876996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.878175 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.878097    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.878620 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.878512    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.880265 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.880220    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.880445 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.880419    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.880520 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.880496    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.882049 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.882021    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.882388 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.882361    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.884206 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.884187    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.939753 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.939717    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49df\" (UniqueName: \"kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:28.950773 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:28.950728    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:29.039638 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.039614    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:33:29.041376 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:29.041348    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ecc50a_f2d6_4670_aa74_5119b82d7dca.slice/crio-311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787 WatchSource:0}: Error finding container 311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787: Status 404 returned error can't find the container with id 311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787
Nov 04 02:33:29.184849 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.184791    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:29.233398 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.233368    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:29.602036 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.601999    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787"}
Nov 04 02:33:29.604449 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.604420    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf" event={"ID":"b675972a-9d65-4a98-9fbe-79b925302105","Type":"ContainerStarted","Data":"7a5aa8f77ef2644c98bfeebe3721a16b67e899a04b932ff296af69c21ef7fb6e"}
Nov 04 02:33:29.606042 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.606008    2573 generic.go:356] "Generic (PLEG): container finished" podID="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" containerID="83261a6390aef62f8b90f7ea9468486bf34f57136fd0b5441bfecd4fca77f505" exitCode=0
Nov 04 02:33:29.606149 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.606126    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cggz4" event={"ID":"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9","Type":"ContainerDied","Data":"83261a6390aef62f8b90f7ea9468486bf34f57136fd0b5441bfecd4fca77f505"}
Nov 04 02:33:29.611853 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.611816    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"]
Nov 04 02:33:29.615126 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.615101    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm" event={"ID":"f7913919-20f9-4b2c-b1fe-81c2552e5c77","Type":"ContainerStarted","Data":"0231cd98ac988f171104641636b7da3246f53c92e6e3789898dabcf8118b5d85"}
Nov 04 02:33:29.615205 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.615161    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm" event={"ID":"f7913919-20f9-4b2c-b1fe-81c2552e5c77","Type":"ContainerStarted","Data":"555ea387eaf93956e9a2bd3687a0c1ec4aaa99f5b122c3549ea5318f0194e2c0"}
Nov 04 02:33:29.624044 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:29.624010    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6381ce_e265_4942_9c1b_0652a4cd88c3.slice/crio-b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa WatchSource:0}: Error finding container b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa: Status 404 returned error can't find the container with id b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa
Nov 04 02:33:29.847347 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:29.847289    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf" podStartSLOduration=2.6970964 podStartE2EDuration="4.847269764s" totalImagesPullingTime="2.150173364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:29.723673741 +0000 UTC m=+101.954857123" watchObservedRunningTime="2026-11-04 02:33:29.847269764 +0000 UTC m=+102.078453148"
Nov 04 02:33:30.111585 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.111505    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.253747 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.253716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.455624 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.455549    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.603342 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.603308    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.622268 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.622238    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cggz4" event={"ID":"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9","Type":"ContainerStarted","Data":"08b885aca0ce47bf5306227deae37ba44bc9ee785f29b1c8b186e1b7ca41e53d"}
Nov 04 02:33:30.622440 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.622299    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cggz4" event={"ID":"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9","Type":"ContainerStarted","Data":"9a42d10b8c72ed74029c63a1bf74be29f601e33b92fb13a7c8d829468a94b8ac"}
Nov 04 02:33:30.623812 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.623779    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa"}
Nov 04 02:33:30.626013 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.625991    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm" event={"ID":"f7913919-20f9-4b2c-b1fe-81c2552e5c77","Type":"ContainerStarted","Data":"0a6ab398715e6413fad5deb3ebe454d8fbda14b7c748586d5ec22549e7cc6b7b"}
Nov 04 02:33:30.737207 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.737134    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"]
Nov 04 02:33:30.739461 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.739437    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:30.743928 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.743888    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cggz4" podStartSLOduration=2.517302282 podStartE2EDuration="4.743870381s" totalImagesPullingTime="2.125329298s" totalInitContainerRuntime="101.238801ms" isStatefulPod=true podCreationTimestamp="2026-11-04 02:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:30.741895323 +0000 UTC m=+102.973078703" watchObservedRunningTime="2026-11-04 02:33:30.743870381 +0000 UTC m=+102.975053767"
Nov 04 02:33:30.754958 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.754934    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.799405 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.799378    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jp2zm\""
Nov 04 02:33:30.799405 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.799397    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 02:33:30.807944 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.807915    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"]
Nov 04 02:33:30.897223 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.897198    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znbgq\" (UniqueName: \"kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:30.897365 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.897230    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:30.910564 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.910530    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:30.916291 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.916269    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bdc474876-v9z8c"]
Nov 04 02:33:30.918197 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.918179    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:30.970630 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.970608    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 02:33:30.970758 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.970746    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 02:33:30.971215 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.971194    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 02:33:30.971331 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.971219    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-afsv71hvcag3u\""
Nov 04 02:33:30.971331 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.971238    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fwnrq\""
Nov 04 02:33:30.971331 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.971287    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 02:33:30.979233 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.979188    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm" podStartSLOduration=2.358269392 podStartE2EDuration="4.979172402s" totalImagesPullingTime="2.62090301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:30.977376294 +0000 UTC m=+103.208559678" watchObservedRunningTime="2026-11-04 02:33:30.979172402 +0000 UTC m=+103.210355788"
Nov 04 02:33:30.981381 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.981359    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bdc474876-v9z8c"]
Nov 04 02:33:30.997924 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.997870    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znbgq\" (UniqueName: \"kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:30.997924 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:30.997904    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:31.000952 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.000920    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:31.059244 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.059217    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znbgq\" (UniqueName: \"kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:31.098784 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.098750    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.098925 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.098815    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.099012 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.098928    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq99h\" (UniqueName: \"kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.099012 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.098987    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.099129 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.099110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.099161 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.099141    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.099231 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.099215    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200280 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200433 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200337    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200559 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200524    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200629 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200610    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200685 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200655    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq99h\" (UniqueName: \"kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200735 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200680    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.200735 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.200701    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.201240 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.201218    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.201614 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.201595    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.203177 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.203153    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.203500 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.203474    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.204270 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.204251    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.264098 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.264055    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq99h\" (UniqueName: \"kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.350694 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.350672    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:31.529934 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.529859    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:31.572243 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.572216    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-djrmx"
Nov 04 02:33:31.631680 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.631649    2573 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="64153097df5bc324244d7d53581209eda2edab5d59db106e2ed37862bdc76d45" exitCode=0
Nov 04 02:33:31.632054 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.631743    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"64153097df5bc324244d7d53581209eda2edab5d59db106e2ed37862bdc76d45"}
Nov 04 02:33:31.691280 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.691252    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:31.691416 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.691306    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:31.696939 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.696917    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:31.730762 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.730738    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"]
Nov 04 02:33:31.772163 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:31.772130    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7767fc65_d7d9_42ce_8787_7b7c9af5ee4f.slice/crio-096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52 WatchSource:0}: Error finding container 096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52: Status 404 returned error can't find the container with id 096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52
Nov 04 02:33:31.821848 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.821778    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:31.909882 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.909855    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bdc474876-v9z8c"]
Nov 04 02:33:31.913530 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:31.913502    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad378d3a_66cb_406a_aa92_ff17ad170512.slice/crio-13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1 WatchSource:0}: Error finding container 13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1: Status 404 returned error can't find the container with id 13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1
Nov 04 02:33:31.964466 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:31.964439    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:32.465734 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.465704    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:33:32.469170 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.469145    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.571119 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.571056    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 02:33:32.572672 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.572646    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:33:32.613083 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613237 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613195    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613311 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613251    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613405 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613379    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613528 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613502    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613648 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613535    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613648 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613632    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.613739 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.613658    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.636487 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.636455    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c" event={"ID":"ad378d3a-66cb-406a-aa92-ff17ad170512","Type":"ContainerStarted","Data":"13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1"}
Nov 04 02:33:32.637815 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.637772    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c" event={"ID":"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f","Type":"ContainerStarted","Data":"096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52"}
Nov 04 02:33:32.642286 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.642264    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:33:32.686326 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.686302    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:32.714476 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714443    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714609 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714609 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714535    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714609 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714605    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714733 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714631    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714733 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714663    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714733 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714868 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.714994 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.714960    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.715446 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.715421    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.715538 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.715467    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.715811 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.715769    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.715850 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.715826    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.717625 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.717598    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.717725 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.717703    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:32.783086 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:32.783019    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:33.082021 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.081995    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:33.124054 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.124013    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:33.314764 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.314695    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:33.461469 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.461433    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:33:33.463720 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.463537    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:33.465553 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:33.465524    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf01f2a0_9285_43ec_9fa8_8bb739ae6a2e.slice/crio-7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4 WatchSource:0}: Error finding container 7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4: Status 404 returned error can't find the container with id 7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4
Nov 04 02:33:33.645214 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.644987    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"9b271b13b984df4f75ab430b293bc31e411427e49d36fd5e395d3c2325b2dd94"}
Nov 04 02:33:33.645214 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.645062    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"b72c337c71219b172ca14474476676f3244d7a48cad2a91fa983ae4e30c0ad8e"}
Nov 04 02:33:33.645214 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.645119    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"6d6b3acd8fa5fe4d0def717d633337735280384b835cf494bab38e171aedfd21"}
Nov 04 02:33:33.648174 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.648143    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768b65b77f-9z9tz" event={"ID":"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e","Type":"ContainerStarted","Data":"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"}
Nov 04 02:33:33.648302 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.648191    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768b65b77f-9z9tz" event={"ID":"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e","Type":"ContainerStarted","Data":"7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4"}
Nov 04 02:33:33.765096 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:33.765030    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-768b65b77f-9z9tz" podStartSLOduration=1.765013808 podStartE2EDuration="1.765013808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:33.764219016 +0000 UTC m=+105.995402411" watchObservedRunningTime="2026-11-04 02:33:33.765013808 +0000 UTC m=+105.996197196"
Nov 04 02:33:34.652513 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.652452    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c" event={"ID":"ad378d3a-66cb-406a-aa92-ff17ad170512","Type":"ContainerStarted","Data":"1bd0b5a30a25ce45ba93416513ed163e3943a44e77769245ba64d21bd164a531"}
Nov 04 02:33:34.655007 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.654593    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c" event={"ID":"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f","Type":"ContainerStarted","Data":"4a5566da0f8cb1ec3c559cb18250f6422ad6f041aee4a7893e71543e69177b7d"}
Nov 04 02:33:34.745093 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.745033    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:34.885005 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.884886    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c" podStartSLOduration=2.090081798 podStartE2EDuration="4.8848713s" totalImagesPullingTime="2.794789502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:34.884463262 +0000 UTC m=+107.115646649" watchObservedRunningTime="2026-11-04 02:33:34.8848713 +0000 UTC m=+107.116054722"
Nov 04 02:33:34.886010 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.885973    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c" podStartSLOduration=2.289879553 podStartE2EDuration="4.88596018s" totalImagesPullingTime="2.596080627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:34.77049437 +0000 UTC m=+107.001677755" watchObservedRunningTime="2026-11-04 02:33:34.88596018 +0000 UTC m=+107.117143560"
Nov 04 02:33:34.889154 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:34.889127    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:35.032881 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.032856    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:35.178002 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.177862    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:35.322239 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.322212    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:35.661385 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.661297    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"e7cca1e604d72ded0ce1c4be348ba56fc8cae6c53ee00e52d7996bfea7923ca5"}
Nov 04 02:33:35.661805 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.661421    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"d0f9c795c74091a8cb2b81e67cec4778b24c87266d2856040bdff60a873709da"}
Nov 04 02:33:35.661805 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.661465    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"355465f32fb82800d4f9ca38adb2c3bc4d95c8ba777cb2d01c93bb4307e99820"}
Nov 04 02:33:35.661805 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.661502    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"839dbba44c2f83d7ba9dcbd8b6c2539079d3abc2c9a0c69ec76339da2c1e22af"}
Nov 04 02:33:35.661805 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.661546    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"379ba94236a643a27bab44298f7088c2bd8e02fd2a1fd23b0552bd21c8684905"}
Nov 04 02:33:35.664246 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.664211    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"5daea0a63f29d353d5e77047431503823076fe8e58dd5b29d15913f2fff42ca7"}
Nov 04 02:33:35.664371 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.664270    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"382d1e62e9a52849b41ffeb5c7ab1baa8e2998998859b7c70133c87dde69d36e"}
Nov 04 02:33:35.664371 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.664301    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" event={"ID":"ab6381ce-e265-4942-9c1b-0652a4cd88c3","Type":"ContainerStarted","Data":"255e38b94444c7450020a4708b67c94d05497f1505091b68064dd50c1accb9cf"}
Nov 04 02:33:35.664562 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.664534    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:35.669151 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.669134    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:33:35.788535 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:35.788484    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg" podStartSLOduration=2.312340587 podStartE2EDuration="7.788471077s" totalImagesPullingTime="5.47613049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:35.787614526 +0000 UTC m=+108.018797911" watchObservedRunningTime="2026-11-04 02:33:35.788471077 +0000 UTC m=+108.019654470"
Nov 04 02:33:36.676822 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:36.676772    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerStarted","Data":"1b49299082e4433965739e29d6808e3c0a9de612881c6738d0a440629211dd4a"}
Nov 04 02:33:36.677297 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:36.676901    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:36.831608 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:36.831551    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.27834086 podStartE2EDuration="8.8315373s" totalImagesPullingTime="6.533913353s" totalInitContainerRuntime="19.283087ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:33:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:33:36.83081196 +0000 UTC m=+109.061995356" watchObservedRunningTime="2026-11-04 02:33:36.8315373 +0000 UTC m=+109.062720683"
Nov 04 02:33:39.197606 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:39.197566    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:39.340000 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:39.339963    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:42.685501 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:42.685475    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:33:43.083379 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:43.083341    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:43.083549 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:43.083414    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:43.088062 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:43.088038    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:43.702651 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:43.702622    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:33:43.872301 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:43.872270    2573 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:33:44.621923 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:44.621888    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:44.786275 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:44.785185    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:44.929305 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:44.929231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:45.299061 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:45.299035    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:45.442169 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:45.442129    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:47.361505 ip-10-0-128-201 kubenswrapper[2573]: E1104 02:33:47.361467    2573 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:33:48.602360 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:48.602327    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:48.746736 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:48.746700    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:51.530117 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:51.530062    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:51.530496 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:51.530139    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:33:52.715475 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:52.715445    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:53.153777 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:53.153741    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:53.299997 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:53.299968    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:33:57.247113 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.247057    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:57.247522 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.247151    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:57.298548 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.298520    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 02:33:57.309902 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.309878    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:57.348176 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.348150    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 02:33:57.348312 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.348295    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:57.360300 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.360275    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:57.399474 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.399452    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 02:33:57.459678 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.459657    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 02:33:57.471182 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.471158    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:57.519955 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.519935    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-82bnw\""
Nov 04 02:33:57.525892 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.525877    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9fmkz\""
Nov 04 02:33:57.528903 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.528888    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:33:57.531013 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.531001    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8zwfm\""
Nov 04 02:33:57.535127 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.535112    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:33:57.540377 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.540362    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:33:57.979643 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:57.979579    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kq82"]
Nov 04 02:33:57.998730 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:57.998704    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a315e6_74d9_46f6_afad_3f9f3d4f98e0.slice/crio-4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373 WatchSource:0}: Error finding container 4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373: Status 404 returned error can't find the container with id 4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373
Nov 04 02:33:58.030577 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:58.030558    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqtjk"]
Nov 04 02:33:58.031947 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:58.031929    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7t4hg"]
Nov 04 02:33:58.032908 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:58.032879    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56433f74_7032_4c74_ab37_3f80599894a4.slice/crio-379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139 WatchSource:0}: Error finding container 379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139: Status 404 returned error can't find the container with id 379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139
Nov 04 02:33:58.033707 ip-10-0-128-201 kubenswrapper[2573]: W1104 02:33:58.033685    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83d912_788d_463f_a790_43d27a3f5765.slice/crio-1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8 WatchSource:0}: Error finding container 1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8: Status 404 returned error can't find the container with id 1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8
Nov 04 02:33:58.753885 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:58.753851    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8kq82" event={"ID":"43a315e6-74d9-46f6-afad-3f9f3d4f98e0","Type":"ContainerStarted","Data":"4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373"}
Nov 04 02:33:58.755053 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:58.755032    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7t4hg" event={"ID":"7a83d912-788d-463f-a790-43d27a3f5765","Type":"ContainerStarted","Data":"1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8"}
Nov 04 02:33:58.756094 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:33:58.756054    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqtjk" event={"ID":"56433f74-7032-4c74-ab37-3f80599894a4","Type":"ContainerStarted","Data":"379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139"}
Nov 04 02:34:01.772061 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:01.772015    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7t4hg" event={"ID":"7a83d912-788d-463f-a790-43d27a3f5765","Type":"ContainerStarted","Data":"3cce304d2b5b21a4690f25e497de7229021b7ade829a873f6c335ef0810156a7"}
Nov 04 02:34:01.772543 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:01.772124    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7t4hg" event={"ID":"7a83d912-788d-463f-a790-43d27a3f5765","Type":"ContainerStarted","Data":"18d44ca069c50b167b506905cdde2bc211618c2b03978d7fcc2d75244aa54261"}
Nov 04 02:34:01.887803 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:01.887745    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7t4hg" podStartSLOduration=131.200345468 podStartE2EDuration="2m13.887728917s" totalImagesPullingTime="2.687383449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:34:01.887290975 +0000 UTC m=+134.118474358" watchObservedRunningTime="2026-11-04 02:34:01.887728917 +0000 UTC m=+134.118912318"
Nov 04 02:34:03.598018 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:03.597968    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:03.793559 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:03.793521    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:04.781566 ip-10-0-128-201 kubenswrapper[2573]: I1104 02:34:04.781458    2573 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:34:04.781651 ip-10-0-128-201 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 02:34:04.799599 ip-10-0-128-201 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 02:34:04.799772 ip-10-0-128-201 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 02:34:04.799951 ip-10-0-128-201 systemd[1]: kubelet.service: Consumed 6.390s CPU time.
Nov 04 02:34:04.815922 ip-10-0-128-201 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 02:34:04.889526 ip-10-0-128-201 kubenswrapper[6019]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:34:04.889526 ip-10-0-128-201 kubenswrapper[6019]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 02:34:04.889526 ip-10-0-128-201 kubenswrapper[6019]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:34:04.889526 ip-10-0-128-201 kubenswrapper[6019]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:34:04.891764 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891686    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.891764 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891761    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.891764 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891766    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.891764 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891770    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891777    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891784    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891788    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891792    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891795    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891799    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891803    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891808    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891811    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891815    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891819    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891827    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891831    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891835    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891839    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891842    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891847    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891851    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.892036 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891855    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891859    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891863    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891867    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891871    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891875    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891879    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891883    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891887    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891891    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891895    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891900    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891904    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891909    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891913    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891917    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891922    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891927    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891931    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891935    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.892919 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891939    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891943    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891947    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891951    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891955    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891960    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891964    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891968    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891972    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891976    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891980    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891985    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891989    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891993    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.891998    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892002    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892007    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892011    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892016    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892021    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.893889 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892025    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892029    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892033    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892046    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892050    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892055    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892059    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892063    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892087    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892094    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892098    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892102    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892106    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892110    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892114    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892118    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892122    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892126    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892131    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892134    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.894796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892139    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892143    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892147    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892152    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892156    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892160    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892165    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892169    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892173    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892177    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892184    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892189    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892194    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892198    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892203    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892208    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892220    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892224    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892228    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.895713 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892236    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892242    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892246    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892251    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.892255    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893559    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893570    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893574    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893579    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893583    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893588    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893592    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893597    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893601    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893605    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893609    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893615    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893620    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893624    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893628    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.896596 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893633    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893637    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893641    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893645    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893649    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893653    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893658    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893662    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893666    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893670    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893676    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893680    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893687    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893691    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893696    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893700    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893704    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893708    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893712    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.897518 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893717    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893721    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893725    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893729    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893734    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893738    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893742    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893746    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893751    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893756    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893760    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893764    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893771    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893778    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893783    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893788    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893793    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893798    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893802    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893806    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.898258 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893811    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893815    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893820    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893824    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893829    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893836    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893841    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893845    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893850    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893854    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893858    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893864    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893869    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893874    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893886    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893890    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893894    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893898    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893902    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.898780 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893905    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893909    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893914    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893918    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893922    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893926    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893930    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893934    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893938    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893942    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893947    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893951    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893955    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893959    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893963    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893967    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893972    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893977    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893982    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893986    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.899268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893990    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893994    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.893998    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894003    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894007    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894010    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894015    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894019    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894023    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894027    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894031    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894036    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894042    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894832    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894841    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894845    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894850    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894855    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894859    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894863    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.899829 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894867    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894872    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894877    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894881    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894885    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894890    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894894    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894898    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894902    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894906    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894911    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894915    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894920    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894924    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894928    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894933    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894937    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894941    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894945    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.900429 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894949    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894957    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894961    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894965    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894970    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894974    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894978    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894983    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894987    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894991    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.894996    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895000    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895004    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895008    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895013    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895017    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895024    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895031    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.901271 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895036    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895040    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895045    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895049    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895053    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895057    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895062    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895086    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895091    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895095    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895100    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895103    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895107    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895112    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895116    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895122    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895127    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895131    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895135    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895139    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.901862 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895144    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895148    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895154    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895158    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895163    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895167    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895171    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895176    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895180    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895184    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895188    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895192    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895198    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895202    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895206    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895210    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895214    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895219    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895223    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.902402 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895228    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895232    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895237    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895241    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895245    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895249    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895253    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895257    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895263    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895268    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895272    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895276    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895280    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895284    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895288    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895294    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895298    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895303    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895307    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895316    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.902876 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895322    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895328    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.895333    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895460    6019 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895483    6019 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895491    6019 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895498    6019 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895505    6019 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895509    6019 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895516    6019 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895522    6019 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895527    6019 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895532    6019 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895538    6019 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895543    6019 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895548    6019 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895553    6019 flags.go:64] FLAG: --cgroup-root=""
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895558    6019 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895562    6019 flags.go:64] FLAG: --client-ca-file=""
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895567    6019 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895574    6019 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895580    6019 flags.go:64] FLAG: --cluster-domain=""
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895585    6019 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895590    6019 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 02:34:04.903393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895595    6019 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895600    6019 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895607    6019 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895613    6019 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895618    6019 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895623    6019 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895628    6019 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895632    6019 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895637    6019 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895642    6019 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895647    6019 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895653    6019 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895658    6019 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895663    6019 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895667    6019 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895672    6019 flags.go:64] FLAG: --enable-server="true"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895676    6019 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895683    6019 flags.go:64] FLAG: --event-burst="100"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895688    6019 flags.go:64] FLAG: --event-qps="50"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895693    6019 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895697    6019 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895702    6019 flags.go:64] FLAG: --eviction-hard=""
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895708    6019 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895713    6019 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895717    6019 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 02:34:04.903983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895722    6019 flags.go:64] FLAG: --eviction-soft=""
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895726    6019 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895731    6019 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895736    6019 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895740    6019 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895744    6019 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895749    6019 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895754    6019 flags.go:64] FLAG: --feature-gates=""
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895761    6019 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895766    6019 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895773    6019 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895778    6019 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895782    6019 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895787    6019 flags.go:64] FLAG: --help="false"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895792    6019 flags.go:64] FLAG: --hostname-override="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895797    6019 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895802    6019 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895807    6019 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895812    6019 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895818    6019 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895822    6019 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895827    6019 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895832    6019 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 02:34:04.904633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895836    6019 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895842    6019 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895848    6019 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895852    6019 flags.go:64] FLAG: --kube-reserved=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895857    6019 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895861    6019 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895866    6019 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895871    6019 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895876    6019 flags.go:64] FLAG: --lock-file=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895880    6019 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895885    6019 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895890    6019 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895899    6019 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895904    6019 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895909    6019 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895913    6019 flags.go:64] FLAG: --logging-format="text"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895918    6019 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895923    6019 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895928    6019 flags.go:64] FLAG: --manifest-url=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895935    6019 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895941    6019 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895947    6019 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895953    6019 flags.go:64] FLAG: --max-pods="110"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895958    6019 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895962    6019 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 02:34:04.905267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895967    6019 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895971    6019 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895976    6019 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895981    6019 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895986    6019 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.895996    6019 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896001    6019 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896006    6019 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896010    6019 flags.go:64] FLAG: --pod-cidr=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896015    6019 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896019    6019 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896025    6019 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896029    6019 flags.go:64] FLAG: --port="10250"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896034    6019 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896038    6019 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0eea7a91cca9de342"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896044    6019 flags.go:64] FLAG: --qos-reserved=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896049    6019 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896054    6019 flags.go:64] FLAG: --register-node="true"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896059    6019 flags.go:64] FLAG: --register-with-taints=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896065    6019 flags.go:64] FLAG: --registry-burst="10"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896089    6019 flags.go:64] FLAG: --registry-qps="5"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896094    6019 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896099    6019 flags.go:64] FLAG: --reserved-memory=""
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896105    6019 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896110    6019 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896114    6019 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 02:34:04.905899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896120    6019 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896125    6019 flags.go:64] FLAG: --runonce="false"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896129    6019 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896134    6019 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896139    6019 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896143    6019 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896148    6019 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896153    6019 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896158    6019 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896162    6019 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896167    6019 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896171    6019 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896176    6019 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896181    6019 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896186    6019 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896191    6019 flags.go:64] FLAG: --system-cgroups=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896195    6019 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896204    6019 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896208    6019 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896212    6019 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896219    6019 flags.go:64] FLAG: --tls-min-version=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896223    6019 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896228    6019 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896233    6019 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896238    6019 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 02:34:04.906581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896242    6019 flags.go:64] FLAG: --v="2"
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896255    6019 flags.go:64] FLAG: --version="false"
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896261    6019 flags.go:64] FLAG: --vmodule=""
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896268    6019 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896273    6019 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896480    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896487    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896494    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896499    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896503    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896507    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896511    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896515    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896520    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896524    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896528    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896535    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896540    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896544    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896548    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.907337 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896552    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896556    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896561    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896565    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896569    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896574    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896578    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896582    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896586    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896591    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896595    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896599    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896603    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896607    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896611    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896615    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896619    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896624    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896628    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896635    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.907909 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896640    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896644    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896649    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896653    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896657    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896662    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896666    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896670    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896676    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896680    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896685    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896689    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896693    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896697    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896701    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896705    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896709    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896714    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896718    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896723    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.908445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896728    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896732    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896736    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896741    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896745    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896749    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896754    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896758    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896762    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896766    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896771    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896777    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896781    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896785    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896789    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896793    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896797    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896802    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896807    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896811    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.908938 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896816    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896820    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896825    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896829    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896833    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896837    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896841    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896845    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896850    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896854    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896858    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896862    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896867    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896871    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896875    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896879    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896884    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896888    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896893    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.909470 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896897    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896904    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896911    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896917    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896924    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896929    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896934    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896938    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896942    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896947    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896951    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.896955    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.896995    6019 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897053    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897061    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.909939 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897083    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897089    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897093    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897097    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897102    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897105    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897109    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897113    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897118    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897122    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897126    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897131    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897135    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897139    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897143    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897149    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897153    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897157    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897161    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897165    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.910445 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897170    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897174    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897181    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897185    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897189    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897193    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897197    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897201    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897205    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897210    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897216    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897220    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897225    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897229    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897234    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897238    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897242    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897246    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897250    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897254    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.911279 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897258    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897262    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897267    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897271    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897275    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897279    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897283    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897287    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897292    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897296    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897301    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897305    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897310    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897314    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897319    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897323    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897328    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897332    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897336    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.911824 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897340    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897344    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897348    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897354    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897359    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897365    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897369    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897374    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897378    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897382    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897386    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897390    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897395    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897399    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897403    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897407    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897411    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897415    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897422    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.912309 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897426    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897430    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897434    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897438    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897443    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897447    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897451    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897455    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897461    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897465    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897469    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897473    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897477    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897481    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897488    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897495    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897503    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897508    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897513    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.912808 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897518    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897523    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897527    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897531    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897535    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897539    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.897544    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.902632    6019 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.902648    6019 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.902680    6019 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.902685    6019 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902743    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902749    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902752    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902755    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902758    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902760    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902763    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902766    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902768    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.913434 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902771    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902774    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902777    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902779    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902782    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902785    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902787    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902790    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902792    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902795    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902797    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902800    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902802    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902805    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902807    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902810    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902813    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902816    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902819    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.913979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902821    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902824    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902827    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902829    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902833    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902837    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902840    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902843    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902845    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902848    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902851    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902854    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902857    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902859    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902861    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902864    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902867    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902869    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902872    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.915024 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902875    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902877    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902881    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902884    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902886    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902889    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902892    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902894    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902897    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902900    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902902    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902905    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902907    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902910    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902912    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902915    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902917    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902920    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902922    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.915626 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902925    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902927    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902930    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902932    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902935    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902937    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902939    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902943    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902945    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902948    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902950    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902953    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902955    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902958    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902961    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902964    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902966    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902969    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902971    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902974    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.916175 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902976    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902979    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902982    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902985    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902987    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902990    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902992    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902994    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902997    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.902999    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903002    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903004    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903007    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903009    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903013    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903016    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903019    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903022    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903025    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.916671 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903027    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.903057    6019 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903208    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903213    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903216    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903219    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903222    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903225    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903228    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903231    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903234    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903236    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903239    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903242    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903244    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903247    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.917203 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903250    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903252    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903254    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903257    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903259    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903262    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903265    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903267    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903270    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903273    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903275    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903278    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903280    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903283    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903285    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903288    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903290    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903293    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903295    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.917632 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903298    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903300    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903303    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903305    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903307    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903310    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903313    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903316    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903318    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903321    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903324    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903327    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903329    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903332    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903334    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903337    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903339    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903342    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903344    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903347    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.918123 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903349    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903352    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903354    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903357    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903359    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903362    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903364    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903367    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903369    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903373    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903376    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903379    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903382    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903385    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903387    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903390    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903392    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903395    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903399    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903401    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.918606 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903404    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903407    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903409    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903412    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903414    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903417    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903419    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903421    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903424    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903426    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903429    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903431    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903434    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903436    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903438    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903441    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903445    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903448    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903451    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.919148 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903454    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903456    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903459    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903461    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903464    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903466    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903469    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903471    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903474    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903476    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903478    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903481    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903484    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903487    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.903510    6019 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903559    6019 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:34:04.919612 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903563    6019 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903566    6019 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903569    6019 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903572    6019 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903575    6019 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903577    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903580    6019 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903583    6019 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903585    6019 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903588    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903590    6019 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903593    6019 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903595    6019 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903598    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903600    6019 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903603    6019 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903605    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903608    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903613    6019 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903616    6019 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:34:04.920013 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903618    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903621    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903624    6019 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903626    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903629    6019 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903631    6019 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903634    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903636    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903639    6019 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903642    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903644    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903647    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903649    6019 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903652    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903654    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903657    6019 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903660    6019 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903662    6019 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903665    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:34:04.920595 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903668    6019 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903671    6019 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903673    6019 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903676    6019 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903678    6019 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903681    6019 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903683    6019 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903686    6019 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903688    6019 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903691    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903693    6019 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903697    6019 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903700    6019 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903702    6019 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903705    6019 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903707    6019 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903710    6019 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903713    6019 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903717    6019 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903720    6019 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:34:04.921087 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903723    6019 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903726    6019 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903728    6019 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903732    6019 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903734    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903737    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903739    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903742    6019 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903744    6019 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903747    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903749    6019 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903752    6019 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903754    6019 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903757    6019 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903759    6019 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903761    6019 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903764    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903766    6019 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903769    6019 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:34:04.921593 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903771    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903774    6019 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903776    6019 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903778    6019 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903782    6019 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903785    6019 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903789    6019 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903792    6019 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903794    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903797    6019 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903800    6019 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903803    6019 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903805    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903808    6019 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903811    6019 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903813    6019 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903816    6019 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903819    6019 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903821    6019 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:34:04.922082 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903824    6019 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903827    6019 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903830    6019 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903833    6019 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903835    6019 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903837    6019 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903840    6019 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:04.903843    6019 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.903897    6019 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.904864    6019 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.904916    6019 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.905108    6019 server.go:1014] "Starting client certificate rotation"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.905255    6019 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.905306    6019 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:26:48 +0000 UTC" deadline="2028-03-31 08:41:29.274439552 +0000 UTC"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.905348    6019 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12318h7m24.369094972s"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.905606    6019 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:34:04.922561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.906698    6019 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:34:04.922963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.908409    6019 log.go:25] "Validated CRI v1 runtime API"
Nov 04 02:34:04.922963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.910351    6019 log.go:25] "Validated CRI v1 image API"
Nov 04 02:34:04.922963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.911134    6019 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 02:34:04.922963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.913457    6019 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7afbd5d7-5cf1-4bc9-b44d-bdce48a4d326:/dev/nvme0n1p4 bf6ab72f-58bd-4cab-a5e7-cd7bad8e30ed:/dev/nvme0n1p3]
Nov 04 02:34:04.923105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.913477    6019 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52/userdata/shm major:0 minor:382 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1/userdata/shm major:0 minor:375 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8/userdata/shm major:0 minor:477 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d82ac74d4b1132a091ccbe4ec9c500a9f52e6cb4b6a2267d52647a49fbb2352/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d82ac74d4b1132a091ccbe4ec9c500a9f52e6cb4b6a2267d52647a49fbb2352/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787/userdata/shm major:0 minor:329 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139/userdata/shm major:0 minor:475 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373/userdata/shm major:0 minor:473 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4/userdata/shm major:0 minor:393 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~projected/kube-api-access-2hbhr:{mountpoint:/var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~projected/kube-api-access-2hbhr major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea/volumes/kubernetes.io~projected/kube-api-access-2d8pz:{mountpoint:/var/lib/kubelet/pods/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea/volumes/kubernetes.io~projected/kube-api-access-2d8pz major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~projected/kube-api-access-cbwgq:{mountpoint:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~projected/kube-api-access-cbwgq major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/33fd8012-9c9e-409e-86c1-527df4c73731/volumes/kubernetes.io~projected/kube-api-access-d6dhp:{mountpoint:/var/lib/kubelet/pods/33fd8012-9c9e-409e-86c1-527df4c73731/volumes/kubernetes.io~projected/kube-api-access-d6dhp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~projected/kube-api-access-fgs97:{mountpoint:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~projected/kube-api-access-fgs97 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/43a315e6-74d9-46f6-afad-3f9f3d4f98e0/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/43a315e6-74d9-46f6-afad-3f9f3d4f98e0/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:471 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~projected/kube-api-access-2zzv2:{mountpoint:/var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~projected/kube-api-access-2zzv2 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:196 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56433f74-7032-4c74-ab37-3f80599894a4/volumes/kubernetes.io~projected/kube-api-access-w8mjs:{mountpoint:/var/lib/kubelet/pods/56433f74-7032-4c74-ab37-3f80599894a4/volumes/kubernetes.io~projected/kube-api-access-w8mjs major:0 minor:472 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe/volumes/kubernetes.io~projected/kube-api-access-tx2cr:{mountpoint:/var/lib/kubelet/pods/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe/volumes/kubernetes.io~projected/kube-api-access-tx2cr major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/634bdde3-359f-4989-8c42-12f216ccfbc8/volumes/kubernetes.io~projected/kube-api-access-ffdn4:{mountpoint:/var/lib/kubelet/pods/634bdde3-359f-4989-8c42-12f216ccfbc8/volumes/kubernetes.io~projected/kube-api-access-ffdn4 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~projected/kube-api-access-gttqw:{mountpoint:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~projected/kube-api-access-gttqw major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~projected/kube-api-access-znbgq:{mountpoint:/var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~projected/kube-api-access-znbgq major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~projected/kube-api-access-2jbmx:{mountpoint:/var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~projected/kube-api-access-2jbmx major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~secret/metrics-certs major:0 minor:467 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~projected/kube-api-access-r9jm6:{mountpoint:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~projected/kube-api-access-r9jm6 major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/alertmanager/9 major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/config-reloader/6 major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~empty-dir/config-out major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/kube-api-access-9469q:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/kube-api-access-9469q major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/tls-assets major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/config-volume major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/web-config major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ba6327-ae4e-46ce-b357-9787f3c1e807/volumes/kubernetes.io~projected/kube-api-access-kdx69:{mountpoint:/var/lib/kubelet/pods/a2ba6327-ae4e-46ce-b357-9787f3c1e807/volumes/kubernetes.io~projected/kube-api-access-kdx69 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~projected/kube-api-access-q49df:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~projected/kube-api-access-q49df major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~projected/kube-api-access-dq99h:{mountpoint:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~projected/kube-api-access-dq99h major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~projected/kube-api-access-w5m6c:{mountpoint:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~projected/kube-api-access-w5m6c major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~projected/kube-api-access-zcszj:{mountpoint:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~projected/kube-api-access-zcszj major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:272 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3cd0132-b993-48e2-8d27-e4a0145bf29d/volumes/kubernetes.io~projected/kube-api-access-wfgkt:{mountpoint:/var/lib/kubelet/pods/d3cd0132-b993-48e2-8d27-e4a0145bf29d/volumes/kubernetes.io~projected/kube-api-access-wfgkt major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~projected/kube-api-access-dplk2:{mountpoint:/var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~projected/kube-api-access-dplk2 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~secret/cert major:0 minor:200 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/def07bfa-aa26-4365-b160-67835f20cbd5/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/def07bfa-aa26-4365-b160-67835f20cbd5/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~projected/kube-api-access-ctfz4:{mountpoint:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~projected/kube-api-access-ctfz4 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~projected/kube-api-access-27k8b:{mountpoint:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~projected/kube-api-access-27k8b major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:247 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/845fa2792a1ccd7fe846539f9ddadf522833ead58f30a6b53ba81be34b6d9861/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/84ab7b8c14985e45dd011922e977a02cd86074e50d99c9d73d42b578ccc2648c/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-103:{mountpoint:/var/lib/containers/storage/overlay/dee7a2024e027b38596d26f524ea5c4f93f277015a12130aeb10a64e0748917d/merged major:0 minor:103 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/83d6da937d94b3e7e9e201e9760451b7dce6360d2f2880b273dd51a259b85208/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/91151d84f768a1024e10f73ab28c2702074b972133bf7b4dac3bbcd1b2b07112/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/6278f0773a7e1c5cb0b28b0b73fac7c0e7817c77e6dd28a53b995d5b5a535058/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-126:{mountpoint:/var/lib/containers/storage/overlay/ea6609fcc94dc3c62a3a9f2971ece55ee0ff7a9b9e30f7807f32d8f3ddb68982/merged major:0 minor:126 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/e5cbb3a85631b0d418f3407baf042c1002b799609f8dc619ec60f2326c05dbb8/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/8487d35fe60d5bf3eda91d307fdae5a3a447854f50fd8f7483ccb0c8f7f3c1da/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-135:{mountpoint:/var/lib/containers/storage/overlay/fcaac3ba64ef87080516116d798b5881e279151c71effa63c906f481b16190ad/merged major:0 minor:135 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/45dcbb27a69866eb4c7302add7f007f5cbca0a86786dd1e222ef231004ff01f7/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/9509d13dbc41194d07168547e6b0e23c9d445637b11950a02af55f20e5fe017f/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/692b0b049a3d51108926aa4c34708ce33b1a46141e53ed0cc8204168f5a94e57/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/d7f41468e5fed9852bb0ea1d54d573b2e7c72ae65eb219ffa5f62c3e20f34da1/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-166:{mountpoint:/var/lib/containers/storage/overlay/8b6f5c9914b1916c256a4ed0801cb06171ea7b4572076c130f647d34b6d464a1/merged major:0 minor:166 fsType:overlay blockSize:0} overlay_0-171:{mountpoint:/var/lib/containers/storage/overlay/b6d84b13c773fc84aed885474995110f36bd23b5f5b8513552a331fc3325c34d/merged major:0 minor:171 fsType:overlay blockSize:0} overlay_0-176:{mountpoint:/var/lib/containers/storage/overlay/650079f9770e4e8ca0b3fe6e1ed92eab0f5bc46ab51426b3c21c8d226c821340/merged major:0 minor:176 fsType:overlay blockSize:0} overlay_0-181:{mountpoint:/var/lib/containers/storage/overlay/525974208c7c20e24cbb8cd0aca08b5d2d41fbd2be8a12fe1ca8882e0c5831ac/merged major:0 minor:181 fsType:overlay blockSize:0} overlay_0-186:{mountpoint:/var/lib/containers/storage/overlay/9b72a411acd0103f9ed923e5c10c2435baaebb50c8dcd8601239a10e8088cdaa/merged major:0 minor:186 fsType:overlay blockSize:0} overlay_0-191:{mountpoint:/var/lib/containers/storage/overlay/899314df82331c4523761a2e1c1b8a1943fa3be456b9637816335b721cf6567a/merged major:0 minor:191 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/e672a536f159f42d08d3b46f7d79ab8b8f970e3a9f07c8f4952bb8a7efcc2b34/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/98f754fbd5d9bb5c1fc64834c7a0aa0034d7dd550ad25c72d0b83eeac983f59e/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/1d2885c8376816a84665e78dbab52f5502c326a050dc96b6ded7fd6863abe64f/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/21c878c9bfba93de42f77b0775d7385cd427f9839f0e4dd539f5bde75754dd93/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/cb755f68605729936b4606699950966ccdd8f0275ea53547de4d48f610d9a8fc/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/18392583af081dbb483c71191a6d28f30e78f511a1ecf69dbaf9a7135971ea47/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/e2ecc7708d51e232b8b04b0b1697c991f2c3db81c31055a9caf1685f6384ba1b/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/5009c6e9b9d729508abab2ad53047d4e65fb3d9a3ef647613790d76d82c80c8d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/1695b491ec4e408876386cc312ea940932ede5c70b640b861a4280693d6cbf94/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/c5569ca9451bcf77903101e35c435cc545e983bfe58826eece9333b7323048d3/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/264515f16d129d10683410096cf42cc0dc969a74f39fa49c21f9985222728ce4/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/ce8ddddbe7ff08fbfee9d9af2eef54a4aef2071651740e9421255ad7fdf6a104/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/c04a39912ec0ab50072c420ff828c2fd0fa9e6e1d9aecadd6263733d9ec1b683/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/a18888b4e8ab7a49d03822981ac230d083013da8c1e7d2ca8971713f1ad6f51b/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/0a7321732503c7bc81156ec84a4c1ba0ae392a1df937f4654eb0d6c324999635/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/91ffa5acce402a19f291cdb41b53b8c9baf54a46e2ef1fab782c8fadb786da64/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/787e19bb25608849eed177d783c1f25536e47dc7af2109d952b1f33aea4a44a0/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/b2a2786ab58884290a49beb720fdc4adc1f7ba5b20e88a5c4ad0808997cbf003/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/c7d4092c37c7408edb1be631bbd237ef4fb557a01a05216284195c07468aa48a/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/f44609619331bec546485d927752222537b9b29acc4910d65821ac6e661fc1df/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/8fcccc5bbf6d6c65f2ea4b9211844f3709182a0b572b4c9c807a45b5b7440d7d/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-341:{mountpoint:/var/lib/containers/storage/overlay/eb3898cddcbd9ef19718b7520aa592a0b65d327f4b2613803c766e2c6a23db5f/merged major:0 minor:341 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/31856a3af48974feb3beb0f5a65da36eed89ff51dc65f517fccedfbe38b7c340/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-349:{mountpoint:/var/lib/containers/storage/overlay/a54e95b0fae23a4cb105768882430e75c21df3a508ab201ff1c6da50c941d3a1/merged major:0 minor:349 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/e1e8f58173e4b14666b182a24212435c4947c804fd0c43a6ee2bbb73b36e6665/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-355:{mountpoint:/var/lib/containers/storage/overlay/5f1735defad3f9b1a4db28357b64712cca6f80cf5ce5d8bb572b9ced386cbbd6/merged major:0 minor:355 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/ec942d7ab4fb4c6ece3d5a2ae2ec3f0b4db3a6e98efd370483b35ed81a0dadeb/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/7a13fc7a189345899b5386c18d4bfc81b2fed18937ae4cf345df7f34892b1e61/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/13ffef77d404e11d29adee6b2dfa8735c4bf7e6a2d026124f10c07c51ae54eda/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/ce9df4612b6738e3c6adfc7e42a1a03f4ea9d0f600c31935a77222c871b6159e/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/39912aa787e0470550e4e3c698ddaf038619e2ccbbc803cf2951677ea417a565/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/3d79033b7b70c63073172d641d8cddad6bf255bca218450876fcf656726427d0/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/e0f3a01f2f83382d05b44de22e917ba20a8e449f1970da7421dceeb6215c1d68/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-406:{mountpoint:/var/lib/containers/storage/overlay/f250c04bdf27781b69e900381b4d93505fbf3861a07203cbf9cb072e65f7544c/merged major:0 minor:406 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/44f91757a37a5128cb5722f8df44bbd56558dd78c5c2f460d08e90b4d634a4e7/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/e82ec8a6e1e46a48e83b7bbe34a561a993f3af435b3114f765c72774fd269ef6/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/7ff191bc4719f658dcdbc46efd394f3d05448be7f12b051f18eca1c2c7cdc5e5/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/7fef127067f38c7a77388295027888e36d1a7eecd2c3a14886f22117078333f5/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/96a71001c5aa7ab0b6741d4be5cd196a55b8231b229faaa01743f5801322c05a/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/7da56e959f7df1003beab27d9497b3f871ecd324ff534212d0859eb45b7da30e/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/43ee0edf3af263aee0083de4e316ace1891d5774e7ff42e3bf71922da5a9b00a/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-447:{mountpoint:/var/lib/containers/storage/overlay/b23ef73ce6cd2f5375165edfd91ddebcfdbffdfef0f3a8330bbedcfb8233d788/merged major:0 minor:447 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/bc953a95832dc58b13fcf63f9a487d776910be9012d449ed8c2dbb592e9b2699/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-452:{mountpoint:/var/lib/containers/storage/overlay/d99bcdf36770d4921ed951324f9ae806e6173a47dd71d8f13fc4ba28ad1ed5bd/merged major:0 minor:452 fsType:overlay blockSize:0} overlay_0-457:{mountpoint:/var/lib/containers/storage/overlay/644901b8fcaf3c26120664c908c880c63bda0f7dcfbd0d49f189d9056a731b4b/merged major:0 minor:457 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/8ecef537f43cf5f3b7c0227d726529b21676ab0111b065b4b96224ff245288f7/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-479:{mountpoint:/var/lib/containers/storage/overlay/b3abc414713aa39dc8efbd60c974364336519ae7f29dc85a2b1023b1ee24684e/merged major:0 minor:479 fsType:overlay blockSize:0} overlay_0-481:{mountpoint:/var/lib/containers/storage/overlay/9e7e231fc5390cfa532694b482cb37dadf656a64a040a300513c4709147c5c6e/merged major:0 minor:481 fsType:overlay blockSize:0} overlay_0-483:{mountpoint:/var/lib/containers/storage/overlay/5a47e01f6d3a52e117b531e5b3ef296c60a519d79a11e0662b71f5fa50e7386c/merged major:0 minor:483 fsType:overlay blockSize:0} overlay_0-485:{mountpoint:/var/lib/containers/storage/overlay/56ccaa32fb8979286d8f8af5bebafe3613bb86a272100011c4fc89d6de87642b/merged major:0 minor:485 fsType:overlay blockSize:0} overlay_0-487:{mountpoint:/var/lib/containers/storage/overlay/390deaec526e99941d656e90d1af623f2c5b5bf47c9440099ab2ddf3f2a8ddb1/merged major:0 minor:487 fsType:overlay blockSize:0} overlay_0-493:{mountpoint:/var/lib/containers/storage/overlay/0400e2a2fece8c0648fe57b6f07f4bf8f347b16cd4526c2bc065b10c40d6be26/merged major:0 minor:493 fsType:overlay blockSize:0} overlay_0-498:{mountpoint:/var/lib/containers/storage/overlay/621e083c0c6725904af84bd10746ab8f135b4bbe3788fe38ea3a9cb347c17cde/merged major:0 minor:498 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/cc6ebecb8b8b7eacc3c697a226d944ed29475630a37845d71b39dd6b36927aa3/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/41b01b3d0353a7276ae985566058e1ae2955875d9707c22b284713931a7a6ac0/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/9c82d81c8afbe8744b97709b7b4bcb70df295bb1919d857a5cad9b3e85f40bd8/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/8b79d7aefd0bce7d701fab3e3de1e7d1984a244780af827ff196a1aba896879a/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c68058d204eaa1c95fa31c44ca5a548903a57a28ab68c7571cef62ede6ba0db2/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/e4b9a30da8fbb3e75916bfcb326f98b0b4a4a7e5e0e06387843c4a863866fa0b/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/e87470578ac27196536b1f0190e3061edb3f16516375bf17b612ac99cbaf0a5c/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/b050edd367eceae4dd96a66e954c21a22dc90eb56c6d4a491b2f4cf1460ca43e/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/851d05899f9b6e1c14889636f5813499e06365d8d52eaae1ebceb9933ad576c6/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/e9afc4b3838cac7a1a355090fc1c855a5ce31e9afe6a4f9e95eb0bad1e299a24/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 02:34:04.924008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.923574    6019 manager.go:217] Machine: {Timestamp:2026-11-04 02:34:04.923101099 +0000 UTC m=+0.064114497 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099597 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e40765fe492fe7f3032e1d976a46e SystemUUID:ec2e4076-5fe4-92fe-7f30-32e1d976a46e BootID:f1c968c5-c6c9-4b5c-8ddb-959c12646422 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-166 DeviceMajor:0 DeviceMinor:166 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-341 DeviceMajor:0 DeviceMinor:341 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-452 DeviceMajor:0 DeviceMinor:452 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:325 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-481 DeviceMajor:0 DeviceMinor:481 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-176 DeviceMajor:0 DeviceMinor:176 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2d70e6ff7d9fa07ecc6fea1a90de088ef1b7b769c52135f27390d2996f25604/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:314 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/634bdde3-359f-4989-8c42-12f216ccfbc8/volumes/kubernetes.io~projected/kube-api-access-ffdn4 DeviceMajor:0 DeviceMinor:52 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~projected/kube-api-access-2jbmx DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:381 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~projected/kube-api-access-w5m6c DeviceMajor:0 DeviceMinor:385 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d84be45f7994e1eaa9ac97105681fb1038e1258f591bb5b013b34efde6a72f53/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~projected/kube-api-access-q49df DeviceMajor:0 DeviceMinor:331 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:319 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/43a315e6-74d9-46f6-afad-3f9f3d4f98e0/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:471 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-479 DeviceMajor:0 DeviceMinor:479 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-485 DeviceMajor:0 DeviceMinor:485 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-135 DeviceMajor:0 DeviceMinor:135 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~projected/kube-api-access-cbwgq DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:272 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:321 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:327 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/33fd8012-9c9e-409e-86c1-527df4c73731/volumes/kubernetes.io~projected/kube-api-access-d6dhp DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/92d363c2cf264f146d79765bbcc9aacc652bcfb1c61ab551f40ae85734d56a31/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-186 DeviceMajor:0 DeviceMinor:186 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-457 DeviceMajor:0 DeviceMinor:457 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f88cfa2fbddb65998b4e18a7c925f8fed14966f95a4d871f8788f5275a1cc029/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:324 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-126 DeviceMajor:0 DeviceMinor:126 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-171 DeviceMajor:0 DeviceMinor:171 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:317 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~projected/kube-api-access-9469q DeviceMajor:0 DeviceMinor:322 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:328 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~projected/kube-api-access-2hbhr DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:323 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:366 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/379ff23329e6698f1a68169bc8d52d5893f21f9591e8002d7072974c5cd8b139/userdata/shm DeviceMajor:0 DeviceMinor:475 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7e5c2365e1386ab5936907d0e6cdb153e222b8e75eaea1016f5d9fe5ef8621d8/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:243 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/096243726c7b616250241722f62c9cd80730a9118d4ddc7a0cbf2f4729c6bf52/userdata/shm DeviceMajor:0 DeviceMinor:382 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4602adc3cee1b5987beab890020e5e67746ed8c81c4423164c51bb587918797b/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:371 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~projected/kube-api-access-dplk2 DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~projected/kube-api-access-ctfz4 DeviceMajor:0 DeviceMinor:289 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:373 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~projected/kube-api-access-dq99h DeviceMajor:0 DeviceMinor:374 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/56433f74-7032-4c74-ab37-3f80599894a4/volumes/kubernetes.io~projected/kube-api-access-w8mjs DeviceMajor:0 DeviceMinor:472 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:224 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~projected/kube-api-access-zcszj DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:318 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13013f29b6549cc409eb784f318832be73b001b46e8b512b49458e07123aacc1/userdata/shm DeviceMajor:0 DeviceMinor:375 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~projected/kube-api-access-gttqw DeviceMajor:0 DeviceMinor:51 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d3cd0132-b993-48e2-8d27-e4a0145bf29d/volumes/kubernetes.io~projected/kube-api-access-wfgkt DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4b31b7344955837a13392ab3892baa834a8f1b7f6423c752b6f86cfaa39c0370/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~projected/kube-api-access-r9jm6 DeviceMajor:0 DeviceMinor:288 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:318 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b055032f9e061e1a2bf3f1790f24fbc3747db40118cf789946e7aa4bfa3c577d/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:315 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:319 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ba6327-ae4e-46ce-b357-9787f3c1e807/volumes/kubernetes.io~projected/kube-api-access-kdx69 DeviceMajor:0 DeviceMinor:53 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:196 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:318 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:318 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-191 DeviceMajor:0 DeviceMinor:191 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:247 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes/kubernetes.io~projected/kube-api-access-27k8b DeviceMajor:0 DeviceMinor:248 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:285 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f7913919-20f9-4b2c-b1fe-81c2552e5c77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:286 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-355 DeviceMajor:0 DeviceMinor:355 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/volumes/kubernetes.io~projected/kube-api-access-znbgq DeviceMajor:0 DeviceMinor:370 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/def07bfa-aa26-4365-b160-67835f20cbd5/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c95cedab4fcbd9ff6337e4a44777c3e924fb3ddc9fcd954d9d55c466a09f799a/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3223c5ed-97b7-46fe-baba-9e54b1b819bd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:219 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fafed2fe72f369faf7fcebb634827be6ef9c83e559b3e2121e2be1842bce1886/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7a83d912-788d-463f-a790-43d27a3f5765/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:467 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bd3056fef3799cdef62e155e1e342112289a6298cbdbe9f8911c91eb131abfc6/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d82ac74d4b1132a091ccbe4ec9c500a9f52e6cb4b6a2267d52647a49fbb2352/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:280 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4821a301aed2438a7896473b122122f91a391e41df733d593ced6aff9db1a373/userdata/shm DeviceMajor:0 DeviceMinor:473 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-487 DeviceMajor:0 DeviceMinor:487 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-406 DeviceMajor:0 DeviceMinor:406 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-181 DeviceMajor:0 DeviceMinor:181 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea/volumes/kubernetes.io~projected/kube-api-access-2d8pz DeviceMajor:0 DeviceMinor:54 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/76b17382-cc28-4fed-8626-5ebb3d863a98/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-103 DeviceMajor:0 DeviceMinor:103 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b06435da2dbf411a62613c308fa32e2a6a02a302819d9d112c92e8cbe98426fa/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-349 DeviceMajor:0 DeviceMinor:349 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f6b308d73ad75e1266c0008ceac58a079a301d0fd3dd8a0c7f3d95edc1945131/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-498 DeviceMajor:0 DeviceMinor:498 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6904f0510a29ffb04dd47696fd5fbf7212c56c7425bb266931b5718e50fccd75/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bfa76fa37f3f11f033a5234eadc28381e9fcc8344217c17928d391fde920d156/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a4f5551-7d57-40db-915a-fa222208f24c/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:384 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4/userdata/shm DeviceMajor:0 DeviceMinor:393 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-447 DeviceMajor:0 DeviceMinor:447 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~projected/kube-api-access-fgs97 DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ad378d3a-66cb-406a-aa92-ff17ad170512/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:372 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1abf570c4dfee462f1160178b04a04b9a034d775c8b4dff7b827810b59bf50c8/userdata/shm DeviceMajor:0 DeviceMinor:477 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe/volumes/kubernetes.io~projected/kube-api-access-tx2cr DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b675972a-9d65-4a98-9fbe-79b925302105/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:276 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:310 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/41ef75a9-a7c5-4c65-9515-0e25b697cfd4/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/db3142b7-fa88-4f52-9f8b-37beedba3df3/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:200 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:316 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:320 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab6381ce-e265-4942-9c1b-0652a4cd88c3/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:326 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-493 DeviceMajor:0 DeviceMinor:493 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53c54c2d-0074-4af2-bbc8-5b5c734fc91b/volumes/kubernetes.io~projected/kube-api-access-2zzv2 DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787/userdata/shm DeviceMajor:0 DeviceMinor:329 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-483 DeviceMajor:0 DeviceMinor:483 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:096243726c7b616 MacAddress:ce:b1:a7:76:ae:1d Speed:10000 Mtu:8901} {Name:13013f29b6549cc MacAddress:4a:a2:56:c4:bd:b5 Speed:10000 Mtu:8901} {Name:1abf570c4dfee46 MacAddress:ae:62:43:c0:70:0c Speed:10000 Mtu:8901} {Name:2d82ac74d4b1132 MacAddress:ea:85:92:3d:5c:d7 Speed:10000 Mtu:8901} {Name:311a869fab1de91 MacAddress:e2:2e:81:2d:f4:62 Speed:10000 Mtu:8901} {Name:379ff23329e6698 MacAddress:06:1b:1e:1b:0d:91 Speed:10000 Mtu:8901} {Name:4821a301aed2438 MacAddress:22:22:19:92:a1:1b Speed:10000 Mtu:8901} {Name:4b31b7344955837 MacAddress:3a:19:40:b2:c0:db Speed:10000 Mtu:8901} {Name:7e5c2365e1386ab MacAddress:ea:93:55:98:be:61 Speed:10000 Mtu:8901} {Name:7f88f1d31e5c94c MacAddress:4a:0d:e9:6d:0d:68 Speed:10000 Mtu:8901} {Name:a2d70e6ff7d9fa0 MacAddress:2e:3e:16:63:62:a1 Speed:10000 Mtu:8901} {Name:b06435da2dbf411 MacAddress:2e:0a:2f:47:7f:f2 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:2c:4e:ff:7e:59 Speed:0 Mtu:9001} {Name:br-int MacAddress:f6:73:be:a7:f7:45 Speed:0 Mtu:8901} {Name:d752ddbd01f838d MacAddress:de:38:10:67:4c:ba Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:2c:4e:ff:7e:59 Speed:0 Mtu:9001} {Name:f6b308d73ad75e1 MacAddress:46:3d:3d:ef:08:2e Speed:10000 Mtu:8901} {Name:fafed2fe72f369f MacAddress:d2:30:e6:87:a4:dd Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:f6:98:ab:73:23:1b Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2e:1c:36:08:18:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 02:34:04.924319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924013    6019 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 02:34:04.924319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924056    6019 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 02:34:04.924425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924397    6019 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 02:34:04.924544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924426    6019 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-201.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 02:34:04.924588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924546    6019 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 02:34:04.924588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924554    6019 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 02:34:04.924588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924564    6019 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:34:04.924588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924584    6019 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:34:04.924689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924668    6019 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 02:34:04.924827 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924817    6019 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 02:34:04.924954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924946    6019 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 02:34:04.924985 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924981    6019 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 02:34:04.925013 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.924997    6019 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 02:34:04.925055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.925016    6019 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 02:34:04.925055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.925030    6019 kubelet.go:405] "Adding apiserver pod source"
Nov 04 02:34:04.925055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.925042    6019 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 02:34:04.925924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.925906    6019 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 02:34:04.925924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.925919    6019 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:34:04.926410 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.926391    6019 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 02:34:04.930365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.929931    6019 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 02:34:04.930365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.929951    6019 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:34:04.930365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930088    6019 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 02:34:04.930506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930388    6019 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 02:34:04.930506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930462    6019 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 02:34:04.930586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930570    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 02:34:04.930834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930612    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 02:34:04.930896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930859    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 02:34:04.930896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930872    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 02:34:04.930896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930882    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 02:34:04.930896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930893    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930904    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930914    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930926    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930936    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930960    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.930976    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.931014    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 02:34:04.931112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.931024    6019 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 02:34:04.934479 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.934462    6019 server.go:1263] "Started kubelet"
Nov 04 02:34:04.934830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.934559    6019 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 02:34:04.934830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.934592    6019 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 02:34:04.934830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.934633    6019 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 02:34:04.935024 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.934930    6019 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 02:34:04.934995 ip-10-0-128-201 systemd[1]: Started Kubernetes Kubelet.
Nov 04 02:34:04.935988 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.935955    6019 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 02:34:04.940107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.939149    6019 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 02:34:04.940107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.939446    6019 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 02:34:04.940107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.939683    6019 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 02:34:04.940107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.939697    6019 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 02:34:04.940107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.939776    6019 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:04.941414    6019 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.941778    6019 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.941791    6019 factory.go:55] Registering systemd factory
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.941801    6019 factory.go:223] Registration of the systemd container factory successfully
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.941886    6019 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:26:49 +0000 UTC" deadline="2028-06-03 02:14:17.186752859 +0000 UTC"
Nov 04 02:34:04.941959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.941922    6019 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13847h40m12.244834787s"
Nov 04 02:34:04.942365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942346    6019 factory.go:153] Registering CRI-O factory
Nov 04 02:34:04.942365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942366    6019 factory.go:223] Registration of the crio container factory successfully
Nov 04 02:34:04.942475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942388    6019 factory.go:103] Registering Raw factory
Nov 04 02:34:04.942475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942402    6019 manager.go:1199] Started watching for new ooms in manager
Nov 04 02:34:04.943786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942750    6019 manager.go:319] Starting recovery of all containers
Nov 04 02:34:04.945348 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.942747    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db" seLinuxMountContext=""
Nov 04 02:34:04.945348 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945303    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.945348 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945340    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945367    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="def07bfa-aa26-4365-b160-67835f20cbd5" volumeName="kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945382    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4f5551-7d57-40db-915a-fa222208f24c" volumeName="kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945401    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41ef75a9-a7c5-4c65-9515-0e25b697cfd4" volumeName="kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945415    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b17382-cc28-4fed-8626-5ebb3d863a98" volumeName="kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945430    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b17382-cc28-4fed-8626-5ebb3d863a98" volumeName="kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945451    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945463    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945482    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945510    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert" seLinuxMountContext=""
Nov 04 02:34:04.948232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945651    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3223c5ed-97b7-46fe-baba-9e54b1b819bd" volumeName="kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls" seLinuxMountContext=""
Nov 04 02:34:04.949516 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.945685    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b17382-cc28-4fed-8626-5ebb3d863a98" volumeName="kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949541    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949571    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949587    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949602    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4f5551-7d57-40db-915a-fa222208f24c" volumeName="kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949622    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="33fd8012-9c9e-409e-86c1-527df4c73731" volumeName="kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949659    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41ef75a9-a7c5-4c65-9515-0e25b697cfd4" volumeName="kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.949743    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="634bdde3-359f-4989-8c42-12f216ccfbc8" volumeName="kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950164    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3cd0132-b993-48e2-8d27-e4a0145bf29d" volumeName="kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950176    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950184    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert" seLinuxMountContext=""
Nov 04 02:34:04.950188 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950193    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3223c5ed-97b7-46fe-baba-9e54b1b819bd" volumeName="kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950201    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="634bdde3-359f-4989-8c42-12f216ccfbc8" volumeName="kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950208    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="634bdde3-359f-4989-8c42-12f216ccfbc8" volumeName="kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950215    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950237    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950244    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950252    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950259    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3cd0132-b993-48e2-8d27-e4a0145bf29d" volumeName="kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950266    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c54c2d-0074-4af2-bbc8-5b5c734fc91b" volumeName="kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950274    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950282    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950289    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950297    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950305    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950312    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950319    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950328    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3223c5ed-97b7-46fe-baba-9e54b1b819bd" volumeName="kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950336    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3223c5ed-97b7-46fe-baba-9e54b1b819bd" volumeName="kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950343    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950350    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950357    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950364    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c54c2d-0074-4af2-bbc8-5b5c734fc91b" volumeName="kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950372    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c54c2d-0074-4af2-bbc8-5b5c734fc91b" volumeName="kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950379    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56433f74-7032-4c74-ab37-3f80599894a4" volumeName="kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950387    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a83d912-788d-463f-a790-43d27a3f5765" volumeName="kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950394    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950402    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950409    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950418    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950425    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950432    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950439    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950455    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3cd0132-b993-48e2-8d27-e4a0145bf29d" volumeName="kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950465    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950472    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950479    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950486    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950494    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950501    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ba6327-ae4e-46ce-b357-9787f3c1e807" volumeName="kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950508    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b675972a-9d65-4a98-9fbe-79b925302105" volumeName="kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950517    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4f5551-7d57-40db-915a-fa222208f24c" volumeName="kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950524    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="33fd8012-9c9e-409e-86c1-527df4c73731" volumeName="kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950531    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41ef75a9-a7c5-4c65-9515-0e25b697cfd4" volumeName="kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950540    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="634bdde3-359f-4989-8c42-12f216ccfbc8" volumeName="kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950550    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b17382-cc28-4fed-8626-5ebb3d863a98" volumeName="kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950561    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950575    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950581    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b675972a-9d65-4a98-9fbe-79b925302105" volumeName="kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950588    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4f5551-7d57-40db-915a-fa222208f24c" volumeName="kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950596    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f18f1cf-0fd4-4c98-8550-ed2539a16bfe" volumeName="kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950603    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a83d912-788d-463f-a790-43d27a3f5765" volumeName="kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950615    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950628    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950650    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" volumeName="kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950657    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b675972a-9d65-4a98-9fbe-79b925302105" volumeName="kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950664    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db3142b7-fa88-4f52-9f8b-37beedba3df3" volumeName="kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950671    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="43a315e6-74d9-46f6-afad-3f9f3d4f98e0" volumeName="kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950678    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" volumeName="kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950686    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950693    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="def07bfa-aa26-4365-b160-67835f20cbd5" volumeName="kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs" seLinuxMountContext=""
Nov 04 02:34:04.950671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950701    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950708    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea" volumeName="kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950715    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7767fc65-d7d9-42ce-8787-7b7c9af5ee4f" volumeName="kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950723    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950731    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950738    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950745    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ba6327-ae4e-46ce-b357-9787f3c1e807" volumeName="kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950752    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab6381ce-e265-4942-9c1b-0652a4cd88c3" volumeName="kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950758    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db3142b7-fa88-4f52-9f8b-37beedba3df3" volumeName="kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950778    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c54c2d-0074-4af2-bbc8-5b5c734fc91b" volumeName="kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950786    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85ecc50a-f2d6-4670-aa74-5119b82d7dca" volumeName="kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950793    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950800    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950808    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f18f1cf-0fd4-4c98-8550-ed2539a16bfe" volumeName="kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950816    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b17382-cc28-4fed-8626-5ebb3d863a98" volumeName="kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950823    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7767fc65-d7d9-42ce-8787-7b7c9af5ee4f" volumeName="kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950832    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad378d3a-66cb-406a-aa92-ff17ad170512" volumeName="kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950838    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b675972a-9d65-4a98-9fbe-79b925302105" volumeName="kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950845    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7913919-20f9-4b2c-b1fe-81c2552e5c77" volumeName="kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950853    6019 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf52ec7-7719-406b-8d32-f91721ec6c02" volumeName="kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b" seLinuxMountContext=""
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950860    6019 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 02:34:04.953316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.950865    6019 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 02:34:04.972633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.972143    6019 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 02:34:04.973906 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.973873    6019 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 02:34:04.973906 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.973903    6019 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 02:34:04.974086 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.973932    6019 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 02:34:04.974086 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:04.973974    6019 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 02:34:04.975936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.975851    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:34:04.976233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.976207    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:34:04.983866 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.983844    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="64153097df5bc324244d7d53581209eda2edab5d59db106e2ed37862bdc76d45" exitCode=0
Nov 04 02:34:04.993128 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.993109    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:34:04.993214 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:04.993141    6019 generic.go:356] "Generic (PLEG): container finished" podID="d3cd0132-b993-48e2-8d27-e4a0145bf29d" containerID="9bf5b17242ab28600adb3ecfc6d2547d21d128a7e096b5ec1ab3b60a7c606484" exitCode=1
Nov 04 02:34:05.003481 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.003460    6019 generic.go:356] "Generic (PLEG): container finished" podID="7ce3cee4-69e7-4743-ae18-ee9921c5c7d9" containerID="83261a6390aef62f8b90f7ea9468486bf34f57136fd0b5441bfecd4fca77f505" exitCode=0
Nov 04 02:34:05.013596 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.013573    6019 generic.go:356] "Generic (PLEG): container finished" podID="1e3d33261584ab125395f2a23dbce857" containerID="cc1ae284d486f2e80e0c696d6004833f192d9b484aa53d3cca8a5fba82fd1b4e" exitCode=0
Nov 04 02:34:05.032702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032682    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="7bc9f86c131521db25f10f0c2af6045c6db514358097f08404e6f7b05a947b62" exitCode=0
Nov 04 02:34:05.032702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032700    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="3927af249fd33b644cef7ab4e4d08956f90816cf027666dc7defb4f3700df365" exitCode=0
Nov 04 02:34:05.032836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032707    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="dca946dd3b8b5a6b267ce078b628a2a7c2f035b66bd88999a2e1689f95165a8c" exitCode=0
Nov 04 02:34:05.032836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032713    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="8632650a710ff7ef0e88a47c8f24ea0d4aea0a80105353cba0f8b621c8fc930f" exitCode=0
Nov 04 02:34:05.032836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032719    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="72475cb38757846b5e0346b2527cf583906d7e8218315f1debb05637649d1800" exitCode=0
Nov 04 02:34:05.032836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.032726    6019 generic.go:356] "Generic (PLEG): container finished" podID="634bdde3-359f-4989-8c42-12f216ccfbc8" containerID="e57c07b2edd7801a79ac11e364d0fdc20661caf53dbf7ee05a2930c919988967" exitCode=0
Nov 04 02:34:05.041845 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.041820    6019 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:34:05.062169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.062124    6019 manager.go:324] Recovery completed
Nov 04 02:34:05.074635 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.074609    6019 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 02:34:05.083695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.083682    6019 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:34:05.085226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.085207    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:34:05.085314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.085241    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:34:05.085314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.085256    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:34:05.088484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088463    6019 cpu_manager.go:226] "Starting" policy="none"
Nov 04 02:34:05.088484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088477    6019 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 02:34:05.088484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088480    6019 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088493    6019 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088599    6019 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088609    6019 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088624    6019 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088629    6019 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088634    6019 policy_none.go:50] "Start"
Nov 04 02:34:05.088640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088643    6019 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 02:34:05.088831 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088653    6019 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 02:34:05.088831 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088765    6019 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 02:34:05.088831 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088772    6019 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 02:34:05.088831 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.088777    6019 policy_none.go:44] "Start"
Nov 04 02:34:05.091115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.091098    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:34:05.091178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.091169    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:34:05.091243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.091186    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:34:05.093484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.093460    6019 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 02:34:05.097004 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.096990    6019 manager.go:341] "Starting Device Plugin manager"
Nov 04 02:34:05.097088 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.097014    6019 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 02:34:05.097088 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097023    6019 server.go:87] "Starting device plugin registration server"
Nov 04 02:34:05.097306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097295    6019 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 02:34:05.097350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097307    6019 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 02:34:05.097418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097402    6019 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 02:34:05.097570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097505    6019 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 02:34:05.097570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097545    6019 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 02:34:05.097570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097552    6019 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 02:34:05.097764 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.097604    6019 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:34:05.097534287Z","UUID":"244c29ee-06d4-41bd-96e7-c03aa0001d61","Handler":null,"Name":"","Endpoint":""}
Nov 04 02:34:05.099176 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.099154    6019 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 02:34:05.099261 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.099192    6019 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-201.us-west-1.compute.internal\" not found"
Nov 04 02:34:05.099667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.099650    6019 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 02:34:05.099756 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.099675    6019 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 02:34:05.136307 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.136288    6019 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:34:05.136779 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.136756    6019 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:34:05.136858 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.136843    6019 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 02:34:05.185024 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.184996    6019 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:34:05.197876 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.197851    6019 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:34:05.198856 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.198839    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:34:05.198942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.198875    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:34:05.198942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.198889    6019 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-201.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:34:05.199048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.198954    6019 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.235683 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.235664    6019 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:34:05.274917 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.274879    6019 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"]
Nov 04 02:34:05.275553 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275533    6019 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.275594 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275555    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerStarted","Data":"ed5b90c73e2152adbe8fd36ef15f7f9275a62d7be6d06332306b81c429933fa6"}
Nov 04 02:34:05.275625 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275595    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerDied","Data":"cc1ae284d486f2e80e0c696d6004833f192d9b484aa53d3cca8a5fba82fd1b4e"}
Nov 04 02:34:05.275625 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275611    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"1e3d33261584ab125395f2a23dbce857","Type":"ContainerStarted","Data":"1de8ff45d43e62a9503d8af7fed06248d7a6ec3e21b9c202ea600a6a944fca0c"}
Nov 04 02:34:05.275625 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275622    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"b41ac5ecba6dc9accfa45177979cc284","Type":"ContainerStarted","Data":"23c05323942e2240bb903058c67007a7c16ee355121a323a729d85cd643edaf4"}
Nov 04 02:34:05.275735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275637    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal" event={"ID":"b41ac5ecba6dc9accfa45177979cc284","Type":"ContainerStarted","Data":"d8fe8d12aab40c51158671e9e8ad8917d56363754b98c99057665c59ab77c9a4"}
Nov 04 02:34:05.275735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.275721    6019 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.337626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.337574    6019 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:34:05.337626 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.337612    6019 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.338169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.338156    6019 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:34:05.338204 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:05.338189    6019 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.346012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.345992    6019 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.346136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.346059    6019 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.437534 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.437477    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.437638 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.437543    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.437638 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.437584    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538729 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538697    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538729 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538708    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538748    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538777    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538824    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41ac5ecba6dc9accfa45177979cc284-config\") pod \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"b41ac5ecba6dc9accfa45177979cc284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.538926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.538827    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3d33261584ab125395f2a23dbce857-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" (UID: \"1e3d33261584ab125395f2a23dbce857\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:05.925853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:05.925824    6019 apiserver.go:52] "Watching apiserver"
Nov 04 02:34:06.039651 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.039625    6019 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:06.039801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.039745    6019 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:06.078701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.078671    6019 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 02:34:06.079395 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.079371    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-jqtjk","kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8","openshift-ovn-kubernetes/ovnkube-node-2g4ht","kube-system/global-pull-secret-syncer-8kq82","openshift-multus/multus-additional-cni-plugins-n8nf4","openshift-image-registry/node-ca-jgvkx","openshift-monitoring/thanos-querier-686fd66bbf-qv4rg","openshift-multus/network-metrics-daemon-7t4hg","openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm","openshift-monitoring/metrics-server-5bdc474876-v9z8c","openshift-dns/node-resolver-xs98j","openshift-monitoring/prometheus-operator-9cd89b74c-czjvg","kube-system/konnectivity-agent-l8xbh","openshift-console/console-768b65b77f-9z9tz","openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf","openshift-monitoring/node-exporter-cggz4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal","openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c","openshift-dns/dns-default-djrmx","openshift-insights/insights-runtime-extractor-r8bmk","openshift-monitoring/alertmanager-main-0","openshift-multus/multus-c7lsk","openshift-cluster-node-tuning-operator/tuned-zrpnk","openshift-console/console-7755b9446c-6hfb2","openshift-network-operator/iptables-alerter-dsbf4","openshift-ingress-canary/ingress-canary-xf8db"]
Nov 04 02:34:06.100654 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.100634    6019 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:34:06.100789 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:06.100673    6019 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:06.101214 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.101199    6019 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:34:06.101271 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:06.101230    6019 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-201.us-west-1.compute.internal"
Nov 04 02:34:06.133828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.133806    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4v2qx\""
Nov 04 02:34:06.134365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134345    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 02:34:06.134526 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134510    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.134599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134549    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.134745 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134725    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fm8qv\""
Nov 04 02:34:06.134745 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134733    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.134850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134786    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.134850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134793    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.134850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134798    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 02:34:06.134850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134805    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 02:34:06.134850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134822    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134869    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134888    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134905    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134952    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-82bnw\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134957    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134969    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134974    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.134971    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 02:34:06.135104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135061    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135184    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8zwfm\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135190    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135215    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rgzgm\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135223    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135251    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lg9c8\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135258    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135316    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135351    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135393    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ffw8m\""
Nov 04 02:34:06.135515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.135427    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6l7l7\""
Nov 04 02:34:06.138630 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.138614    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.138684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.138659    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vd7gf\""
Nov 04 02:34:06.138684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.138670    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 02:34:06.138821 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.138808    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 02:34:06.138867 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.138826    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 02:34:06.139030 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139013    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 02:34:06.139121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139107    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.139166 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139114    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.139166 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139122    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.139255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139215    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.139255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139229    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gjd4x\""
Nov 04 02:34:06.139255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139234    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.139255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139245    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9fmkz\""
Nov 04 02:34:06.139391 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139257    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-l8p2p\""
Nov 04 02:34:06.139391 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.139262    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmwv6\""
Nov 04 02:34:06.141007 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.140993    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.141141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.141129    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.141770 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.141758    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 02:34:06.142057 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142040    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.142132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142085    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.142132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142107    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.142221 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142143    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.142221 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142168    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8pz\" (UniqueName: \"kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.142221 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142206    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.142373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142247    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.142373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142293    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt2vt\""
Nov 04 02:34:06.142373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142338    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.142373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142349    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142338    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-textfile\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142368    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142409    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142424    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qw9gk\""
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142546    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142579    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.142628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142594    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6tfxk\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142639    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142669    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142685    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142695    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142675    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142752    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142772    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142811    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142820    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142793    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142848    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142875    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142901    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142906    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142921    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cni-binary-copy\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142927    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.142979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.142951    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143021    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143044    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143098    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143119    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctfz4\" (UniqueName: \"kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143277    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143185    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143604    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143683    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143721    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143725    6019 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 02:34:06.143788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143742    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zzv2\" (UniqueName: \"kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143804    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-tuned\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143852    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143887    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143912    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143949    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.143970    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144011    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144046    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144090    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144117    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144118    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b675972a-9d65-4a98-9fbe-79b925302105-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144183    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-data-volume\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144215    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jbmx\" (UniqueName: \"kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144261    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144297    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144322    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144395    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144422    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7913919-20f9-4b2c-b1fe-81c2552e5c77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144426    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144440    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ad378d3a-66cb-406a-aa92-ff17ad170512-audit-log\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144457    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144481    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144582    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db3142b7-fa88-4f52-9f8b-37beedba3df3-cert\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144587    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jm6\" (UniqueName: \"kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144608    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144635    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-script-lib\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144623    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144669    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144694    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144722    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144753    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144775    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144800    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144852    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144876    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144899    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144921    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144919    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4f5551-7d57-40db-915a-fa222208f24c-tmp-dir\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144960    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33fd8012-9c9e-409e-86c1-527df4c73731-iptables-alerter-script\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.144968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144961    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144983    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-ovnkube-config\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.144992    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145030    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145061    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145113    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145161    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145188    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145206    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6381ce-e265-4942-9c1b-0652a4cd88c3-tmp\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145215    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145241    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145265    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145295    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145304    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76b17382-cc28-4fed-8626-5ebb3d863a98-env-overrides\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145320    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145324    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145374    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6dhp\" (UniqueName: \"kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145401    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145432    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145458    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145484    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dplk2\" (UniqueName: \"kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145509    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145542    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145581    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145634    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/634bdde3-359f-4989-8c42-12f216ccfbc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145663    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145680    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49df\" (UniqueName: \"kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145696    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145710    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145718    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2ba6327-ae4e-46ce-b357-9787f3c1e807-tmp-dir\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145726    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145719    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-daemon-config\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145754    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145780    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145866    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145904    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.145901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145922    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145935    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdn4\" (UniqueName: \"kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145969    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.145993    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a83d912-788d-463f-a790-43d27a3f5765-metrics-certs\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146010    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146030    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146047    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146063    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146100    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146124    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146153    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcszj\" (UniqueName: \"kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146169    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgs97\" (UniqueName: \"kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146183    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146185    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146220    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146237    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146262    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146277    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146334    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-serviceca\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146369    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146402    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146414    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146462    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2cr\" (UniqueName: \"kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146493    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146521    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146554    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146580    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146643    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbhr\" (UniqueName: \"kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146668    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146683    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146717    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6381ce-e265-4942-9c1b-0652a4cd88c3-metrics-client-ca\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146735    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq99h\" (UniqueName: \"kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146764    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146794    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a4f5551-7d57-40db-915a-fa222208f24c-metrics-tls\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146861    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146886    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146905    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:34:06.147357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146959    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.146988    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147638    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-original-pull-secret\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147637    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbwgq\" (UniqueName: \"kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147675    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147696    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147731    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgkt\" (UniqueName: \"kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147760    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147789    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147826    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-tmp\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147846    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147879    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147908    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147934    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147950    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.147966    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148004    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148125    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148162    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148190    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148223    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148227    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76b17382-cc28-4fed-8626-5ebb3d863a98-ovn-node-metrics-cert\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148262    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdx69\" (UniqueName: \"kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.148456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.148130    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.153233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.153218    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-metrics-client-ca\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.153277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.153264    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3223c5ed-97b7-46fe-baba-9e54b1b819bd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.153306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.153266    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4f5551-7d57-40db-915a-fa222208f24c-config-volume\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.153306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.153266    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3223c5ed-97b7-46fe-baba-9e54b1b819bd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.201243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.201189    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8pz\" (UniqueName: \"kubernetes.io/projected/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kube-api-access-2d8pz\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.201484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.201469    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctfz4\" (UniqueName: \"kubernetes.io/projected/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-api-access-ctfz4\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.202257 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.202233    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zzv2\" (UniqueName: \"kubernetes.io/projected/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-kube-api-access-2zzv2\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.202487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.202467    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/76b17382-cc28-4fed-8626-5ebb3d863a98-kube-api-access-gttqw\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.202575 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.202471    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jbmx\" (UniqueName: \"kubernetes.io/projected/7a83d912-788d-463f-a790-43d27a3f5765-kube-api-access-2jbmx\") pod \"network-metrics-daemon-7t4hg\" (UID: \"7a83d912-788d-463f-a790-43d27a3f5765\") " pod="openshift-multus/network-metrics-daemon-7t4hg"
Nov 04 02:34:06.202719 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.202701    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jm6\" (UniqueName: \"kubernetes.io/projected/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-kube-api-access-r9jm6\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.202944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.202925    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6dhp\" (UniqueName: \"kubernetes.io/projected/33fd8012-9c9e-409e-86c1-527df4c73731-kube-api-access-d6dhp\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.209845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.209824    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcszj\" (UniqueName: \"kubernetes.io/projected/b675972a-9d65-4a98-9fbe-79b925302105-kube-api-access-zcszj\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.209919 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.209888    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dplk2\" (UniqueName: \"kubernetes.io/projected/db3142b7-fa88-4f52-9f8b-37beedba3df3-kube-api-access-dplk2\") pod \"ingress-canary-xf8db\" (UID: \"db3142b7-fa88-4f52-9f8b-37beedba3df3\") " pod="openshift-ingress-canary/ingress-canary-xf8db"
Nov 04 02:34:06.210373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210339    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49df\" (UniqueName: \"kubernetes.io/projected/ab6381ce-e265-4942-9c1b-0652a4cd88c3-kube-api-access-q49df\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.210592 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210574    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbhr\" (UniqueName: \"kubernetes.io/projected/1a4f5551-7d57-40db-915a-fa222208f24c-kube-api-access-2hbhr\") pod \"dns-default-djrmx\" (UID: \"1a4f5551-7d57-40db-915a-fa222208f24c\") " pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:06.210666 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210608    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2cr\" (UniqueName: \"kubernetes.io/projected/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-kube-api-access-tx2cr\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.210723 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210710    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgs97\" (UniqueName: \"kubernetes.io/projected/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-kube-api-access-fgs97\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.210809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210784    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdn4\" (UniqueName: \"kubernetes.io/projected/634bdde3-359f-4989-8c42-12f216ccfbc8-kube-api-access-ffdn4\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.210869 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.210852    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdx69\" (UniqueName: \"kubernetes.io/projected/a2ba6327-ae4e-46ce-b357-9787f3c1e807-kube-api-access-kdx69\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.211040 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.211023    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbwgq\" (UniqueName: \"kubernetes.io/projected/3223c5ed-97b7-46fe-baba-9e54b1b819bd-kube-api-access-cbwgq\") pod \"prometheus-operator-9cd89b74c-czjvg\" (UID: \"3223c5ed-97b7-46fe-baba-9e54b1b819bd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-czjvg"
Nov 04 02:34:06.211118 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.211045    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgkt\" (UniqueName: \"kubernetes.io/projected/d3cd0132-b993-48e2-8d27-e4a0145bf29d-kube-api-access-wfgkt\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.211363 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.211342    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mjs\" (UniqueName: \"kubernetes.io/projected/56433f74-7032-4c74-ab37-3f80599894a4-kube-api-access-w8mjs\") pod \"network-check-target-jqtjk\" (UID: \"56433f74-7032-4c74-ab37-3f80599894a4\") " pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:34:06.211394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.211375    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq99h\" (UniqueName: \"kubernetes.io/projected/ad378d3a-66cb-406a-aa92-ff17ad170512-kube-api-access-dq99h\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.233184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233163    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 02:34:06.233184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233163    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 02:34:06.233347 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233334    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 02:34:06.233392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233376    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 02:34:06.233544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233532    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 02:34:06.233590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233566    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-client-certs\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.233890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233875    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 02:34:06.233890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.233883    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6g9rb\""
Nov 04 02:34:06.234022 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234010    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 02:34:06.234098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234052    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 02:34:06.234098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234056    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 02:34:06.234192 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234158    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 02:34:06.234238 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234192    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mpgnq\""
Nov 04 02:34:06.234268 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234239    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m7pds\""
Nov 04 02:34:06.234295 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234270    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 02:34:06.234326 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234304    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 02:34:06.234326 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234315    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 02:34:06.234378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234324    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 02:34:06.234378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234315    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 02:34:06.234378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234363    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 02:34:06.234457 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234415    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 02:34:06.234457 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234452    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 02:34:06.234960 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234893    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-afsv71hvcag3u\""
Nov 04 02:34:06.235047 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.234992    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.235146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.235125    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.235247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.235230    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 02:34:06.235375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.235355    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.235464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.235442    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 02:34:06.235538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.235521    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.236747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.236728    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.236873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.236854    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.237097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237061    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d2bo7iqmu07ta\""
Nov 04 02:34:06.237149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237127    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ad378d3a-66cb-406a-aa92-ff17ad170512-metrics-server-audit-profiles\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.237420 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237405    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pf5mr\""
Nov 04 02:34:06.237450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237419    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 02:34:06.237558 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237541    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-client-ca-bundle\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.237751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237739    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 02:34:06.237829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.237805    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238140    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238336    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238364    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238402    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238617    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ad378d3a-66cb-406a-aa92-ff17ad170512-secret-metrics-server-tls\") pod \"metrics-server-5bdc474876-v9z8c\" (UID: \"ad378d3a-66cb-406a-aa92-ff17ad170512\") " pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238693    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gd7d8\""
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238829    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 02:34:06.238891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238837    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jp2zm\""
Nov 04 02:34:06.239308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.238910    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 02:34:06.239308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239125    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 02:34:06.239308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239258    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 02:34:06.239308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239303    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 02:34:06.240097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239467    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 02:34:06.240097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239612    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 02:34:06.240097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.239711    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 02:34:06.240097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.240042    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gc8m9\""
Nov 04 02:34:06.241191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.240522    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fwnrq\""
Nov 04 02:34:06.241191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.241116    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 02:34:06.242121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.242102    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 02:34:06.243241 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243222    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.243376 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243256    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.243376 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243280    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.243376 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243286    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.243376 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243329    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.243519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.243388    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"console-7755b9446c-6hfb2\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") " pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:06.245512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.245493    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7913919-20f9-4b2c-b1fe-81c2552e5c77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-95sfm\" (UID: \"f7913919-20f9-4b2c-b1fe-81c2552e5c77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-95sfm"
Nov 04 02:34:06.246689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.246674    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.246735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.246677    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.247902 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.247888    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-grpc-tls\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.248642 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248624    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab6381ce-e265-4942-9c1b-0652a4cd88c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686fd66bbf-qv4rg\" (UID: \"ab6381ce-e265-4942-9c1b-0652a4cd88c3\") " pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:06.248681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248624    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b675972a-9d65-4a98-9fbe-79b925302105-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zvwwf\" (UID: \"b675972a-9d65-4a98-9fbe-79b925302105\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zvwwf"
Nov 04 02:34:06.248681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248634    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-tls\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.248739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248689    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.248739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248711    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.248739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248735    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248766    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248771    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-k8s-cni-cncf-io\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248768    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-etc-kubernetes\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248788    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248803    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248817    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.248826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248822    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-socket-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248835    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248853    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-var-lib-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248859    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248877    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-log-socket\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248885    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248880    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248915    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248923    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-socket-dir-parent\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248942    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248962    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248966    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.248985    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2ba6327-ae4e-46ce-b357-9787f3c1e807-hosts-file\") pod \"node-resolver-xs98j\" (UID: \"a2ba6327-ae4e-46ce-b357-9787f3c1e807\") " pod="openshift-dns/node-resolver-xs98j"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249007    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249015    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-node-exporter-wtmp\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249019    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-kubernetes\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249042    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249058    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249107    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-run\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249110    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249146    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249100    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-sys-fs\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249172    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249187    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249202    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249218    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249231    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249244    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249272    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-slash\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249289    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249297    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249308    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249319    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53c54c2d-0074-4af2-bbc8-5b5c734fc91b-crio-socket\") pod \"insights-runtime-extractor-r8bmk\" (UID: \"53c54c2d-0074-4af2-bbc8-5b5c734fc91b\") " pod="openshift-insights/insights-runtime-extractor-r8bmk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249323    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249340    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-systemd\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249341    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-multus\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249360    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249368    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249414    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-multus-certs\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249442    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249446    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249499    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249508    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-cnibin\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249527    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249532    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249536    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33fd8012-9c9e-409e-86c1-527df4c73731-host-slash\") pod \"iptables-alerter-dsbf4\" (UID: \"33fd8012-9c9e-409e-86c1-527df4c73731\") " pod="openshift-network-operator/iptables-alerter-dsbf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249551    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znbgq\" (UniqueName: \"kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249578    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249589    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249600    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249652    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249677    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249721    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249727    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-kubelet\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249725    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-system-cni-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249739    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249767    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:34:06.249781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249802    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249827    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-conf\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249828    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249864    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249864    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249882    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249917    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249926    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-modprobe-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249953    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/def07bfa-aa26-4365-b160-67835f20cbd5-agent-certs\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249953    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249966    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/def07bfa-aa26-4365-b160-67835f20cbd5-konnectivity-ca\") pod \"konnectivity-agent-l8xbh\" (UID: \"def07bfa-aa26-4365-b160-67835f20cbd5\") " pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249972    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-sys\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249977    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-bin\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249941    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.249993    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250006    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250029    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250052    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250081    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-kubelet\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250092    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250110    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250131    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250153    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250154    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysctl-d\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250183    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250178    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250212    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-cnibin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250376    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250384    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250418    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250443    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250454    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-run-netns\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250444    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250468    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250493    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-host-cni-netd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250496    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250517    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-hostroot\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250526    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250549    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250528    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/634bdde3-359f-4989-8c42-12f216ccfbc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n8nf4\" (UID: \"634bdde3-359f-4989-8c42-12f216ccfbc8\") " pod="openshift-multus/multus-additional-cni-plugins-n8nf4"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250572    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250614    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250623    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-var-lib-cni-bin\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250630    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-systemd-units\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250634    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250670    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250692    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250700    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250735    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250742    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-sys\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250729    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-etc-sysconfig\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250758    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250779    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-systemd\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250789    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250800    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-host-run-netns\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250813    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250833    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-multus-conf-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250840    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250863    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f18f1cf-0fd4-4c98-8550-ed2539a16bfe-host\") pod \"node-ca-jgvkx\" (UID: \"5f18f1cf-0fd4-4c98-8550-ed2539a16bfe\") " pod="openshift-image-registry/node-ca-jgvkx"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250868    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250904    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250910    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250932    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250931    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250958    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250973    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-dbus\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.250984    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251006    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251007    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251015    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-node-log\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251016    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-registration-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251039    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251052    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-host\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251061    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251063    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea-device-dir\") pod \"aws-ebs-csi-driver-node-qfpz8\" (UID: \"1fa40eff-cd47-4cf7-98f9-f6ceb6d511ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfpz8"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251113    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251125    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3cd0132-b993-48e2-8d27-e4a0145bf29d-system-cni-dir\") pod \"multus-c7lsk\" (UID: \"d3cd0132-b993-48e2-8d27-e4a0145bf29d\") " pod="openshift-multus/multus-c7lsk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251131    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251153    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251171    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251176    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43a315e6-74d9-46f6-afad-3f9f3d4f98e0-kubelet-config\") pod \"global-pull-secret-syncer-8kq82\" (UID: \"43a315e6-74d9-46f6-afad-3f9f3d4f98e0\") " pod="kube-system/global-pull-secret-syncer-8kq82"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251197    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251197    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251219    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-var-lib-kubelet\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251230    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251249    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41ef75a9-a7c5-4c65-9515-0e25b697cfd4-lib-modules\") pod \"tuned-zrpnk\" (UID: \"41ef75a9-a7c5-4c65-9515-0e25b697cfd4\") " pod="openshift-cluster-node-tuning-operator/tuned-zrpnk"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251258    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251283    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.251240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251221    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-etc-openvswitch\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251198    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ce3cee4-69e7-4743-ae18-ee9921c5c7d9-root\") pod \"node-exporter-cggz4\" (UID: \"7ce3cee4-69e7-4743-ae18-ee9921c5c7d9\") " pod="openshift-monitoring/node-exporter-cggz4"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251308    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251324    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76b17382-cc28-4fed-8626-5ebb3d863a98-run-ovn\") pod \"ovnkube-node-2g4ht\" (UID: \"76b17382-cc28-4fed-8626-5ebb3d863a98\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251383    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251410    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.253590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.251482    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.307532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.307511    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"console-768b65b77f-9z9tz\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") " pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:06.307800 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.307781    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znbgq\" (UniqueName: \"kubernetes.io/projected/7767fc65-d7d9-42ce-8787-7b7c9af5ee4f-kube-api-access-znbgq\") pod \"monitoring-plugin-77f4f76594-4jh5c\" (UID: \"7767fc65-d7d9-42ce-8787-7b7c9af5ee4f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:34:06.308371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.308356    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"alertmanager-main-0\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:06.602456 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:06.602423    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:34:06.616798 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:06.616777    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4jh5c"
Nov 04 02:34:07.188747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:07.188720    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-djrmx"
Nov 04 02:34:07.204676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:07.204654    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:34:09.435931 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:09.435903    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-686fd66bbf-qv4rg"
Nov 04 02:34:09.765022 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:09.764991    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:11.216434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:11.216398    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:34:12.414757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:12.414729    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bdc474876-v9z8c"
Nov 04 02:34:13.242835 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:13.242802    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:13.243032 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:13.243010    6019 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 02:34:15.018327 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:15.018300    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8xbh"
Nov 04 02:34:16.021687 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:16.021660    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g4ht"
Nov 04 02:34:18.783437 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:18.783400    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:18.925198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:18.925165    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:24.272730 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:24.272693    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:31.110184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.110140    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7755b9446c-6hfb2" podUID="fcf52ec7-7719-406b-8d32-f91721ec6c02" containerName="console" containerID="cri-o://50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd" gracePeriod=15
Nov 04 02:34:31.352872 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.352851    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:31.448408 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448345    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448412    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448564    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448606    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448633    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448677    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448710    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") pod \"fcf52ec7-7719-406b-8d32-f91721ec6c02\" (UID: \"fcf52ec7-7719-406b-8d32-f91721ec6c02\") "
Nov 04 02:34:31.448836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448735    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:34:31.448958 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448928    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:34:31.449023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.448957    6019 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-oauth-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.449062 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.449022    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:34:31.449450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.449424    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:34:31.450953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.450920    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "kube-api-access-27k8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:34:31.450953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.450938    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:31.451151 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.450977    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert" pod "fcf52ec7-7719-406b-8d32-f91721ec6c02" (UID: "fcf52ec7-7719-406b-8d32-f91721ec6c02"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:31.550234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550205    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27k8b\" (UniqueName: \"kubernetes.io/projected/fcf52ec7-7719-406b-8d32-f91721ec6c02-kube-api-access-27k8b\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.550234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550229    6019 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.550234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550239    6019 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-oauth-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.550428 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550248    6019 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.550428 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550257    6019 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf52ec7-7719-406b-8d32-f91721ec6c02-console-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:31.550428 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:31.550269    6019 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf52ec7-7719-406b-8d32-f91721ec6c02-tmp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:32.131705 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.131675    6019 generic.go:356] "Generic (PLEG): container finished" podID="fcf52ec7-7719-406b-8d32-f91721ec6c02" containerID="50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd" exitCode=0
Nov 04 02:34:32.132064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.131722    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b9446c-6hfb2"
Nov 04 02:34:32.132064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.131747    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b9446c-6hfb2" event={"ID":"fcf52ec7-7719-406b-8d32-f91721ec6c02","Type":"ContainerDied","Data":"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"}
Nov 04 02:34:32.132064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.131786    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b9446c-6hfb2" event={"ID":"fcf52ec7-7719-406b-8d32-f91721ec6c02","Type":"ContainerDied","Data":"d752ddbd01f838ddbffdf7a1e9a1c5d1f9bd326c39061a7d87bfbd5b35797c34"}
Nov 04 02:34:32.132064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.131804    6019 scope.go:122] "RemoveContainer" containerID="50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"
Nov 04 02:34:32.141795 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.141714    6019 scope.go:122] "RemoveContainer" containerID="50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"
Nov 04 02:34:32.142184 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:32.142103    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd\": container with ID starting with 50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd not found: ID does not exist" containerID="50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"
Nov 04 02:34:32.142184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.142137    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd"} err="failed to get container status \"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd\": rpc error: code = NotFound desc = could not find container \"50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd\": container with ID starting with 50d00270e2a77967e48f179339e53c7216475914dd0d0a7c7cb5aed18f6bc1cd not found: ID does not exist"
Nov 04 02:34:32.294740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.294713    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:34:32.344413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.344385    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7755b9446c-6hfb2"]
Nov 04 02:34:32.980697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:32.980653    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcf52ec7-7719-406b-8d32-f91721ec6c02" path="/var/lib/kubelet/pods/fcf52ec7-7719-406b-8d32-f91721ec6c02/volumes"
Nov 04 02:34:36.601713 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:34:36.601668    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:34:41.219209 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:41.219135    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jqtjk"
Nov 04 02:34:49.868165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:49.868131    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:50.216290 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:50.216207    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:34:52.080224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080194    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080472    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="alertmanager" containerID="cri-o://379ba94236a643a27bab44298f7088c2bd8e02fd2a1fd23b0552bd21c8684905" gracePeriod=120
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080514    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy" containerID="cri-o://d0f9c795c74091a8cb2b81e67cec4778b24c87266d2856040bdff60a873709da" gracePeriod=120
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080513    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-web" containerID="cri-o://355465f32fb82800d4f9ca38adb2c3bc4d95c8ba777cb2d01c93bb4307e99820" gracePeriod=120
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080534    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="config-reloader" containerID="cri-o://839dbba44c2f83d7ba9dcbd8b6c2539079d3abc2c9a0c69ec76339da2c1e22af" gracePeriod=120
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080574    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-metric" containerID="cri-o://e7cca1e604d72ded0ce1c4be348ba56fc8cae6c53ee00e52d7996bfea7923ca5" gracePeriod=120
Nov 04 02:34:52.080650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.080567    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="prom-label-proxy" containerID="cri-o://1b49299082e4433965739e29d6808e3c0a9de612881c6738d0a440629211dd4a" gracePeriod=120
Nov 04 02:34:52.210882 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210853    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="1b49299082e4433965739e29d6808e3c0a9de612881c6738d0a440629211dd4a" exitCode=0
Nov 04 02:34:52.210882 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210877    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="d0f9c795c74091a8cb2b81e67cec4778b24c87266d2856040bdff60a873709da" exitCode=0
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210886    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="355465f32fb82800d4f9ca38adb2c3bc4d95c8ba777cb2d01c93bb4307e99820" exitCode=0
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210908    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="839dbba44c2f83d7ba9dcbd8b6c2539079d3abc2c9a0c69ec76339da2c1e22af" exitCode=0
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210914    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="379ba94236a643a27bab44298f7088c2bd8e02fd2a1fd23b0552bd21c8684905" exitCode=0
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210935    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"1b49299082e4433965739e29d6808e3c0a9de612881c6738d0a440629211dd4a"}
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.210987    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"d0f9c795c74091a8cb2b81e67cec4778b24c87266d2856040bdff60a873709da"}
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.211011    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"355465f32fb82800d4f9ca38adb2c3bc4d95c8ba777cb2d01c93bb4307e99820"}
Nov 04 02:34:52.211036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.211034    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"839dbba44c2f83d7ba9dcbd8b6c2539079d3abc2c9a0c69ec76339da2c1e22af"}
Nov 04 02:34:52.211266 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:52.211061    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"379ba94236a643a27bab44298f7088c2bd8e02fd2a1fd23b0552bd21c8684905"}
Nov 04 02:34:53.219111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.219058    6019 generic.go:356] "Generic (PLEG): container finished" podID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerID="e7cca1e604d72ded0ce1c4be348ba56fc8cae6c53ee00e52d7996bfea7923ca5" exitCode=0
Nov 04 02:34:53.219111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.219111    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"e7cca1e604d72ded0ce1c4be348ba56fc8cae6c53ee00e52d7996bfea7923ca5"}
Nov 04 02:34:53.329178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329156    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:53.329385 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329365    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329411    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329446    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329498 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329474    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329564 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329547    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329618 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329591    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329669 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329620    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329669 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329661    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.329786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329745    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:34:53.329951 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.329927    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.330017 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330004    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.330094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330008    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:34:53.330358 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330046    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.330358 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330266    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.330358 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330305    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") pod \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\" (UID: \"85ecc50a-f2d6-4670-aa74-5119b82d7dca\") "
Nov 04 02:34:53.330537 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330519    6019 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-main-db\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.330592 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330544    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-metrics-client-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.331225 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.330913    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:34:53.333249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.333211    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:34:53.333585 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.333551    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.333723 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.333699    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.334101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.334031    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.334191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.334144    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:34:53.334282 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.334264    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.334587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.334551    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.334861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.334838    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "kube-api-access-9469q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:34:53.338302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.338281    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.344495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.344474    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config" pod "85ecc50a-f2d6-4670-aa74-5119b82d7dca" (UID: "85ecc50a-f2d6-4670-aa74-5119b82d7dca"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:34:53.431159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431098    6019 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-tls-assets\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431122    6019 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-out\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431136    6019 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-web-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431151    6019 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431167    6019 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85ecc50a-f2d6-4670-aa74-5119b82d7dca-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431181    6019 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431194    6019 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431207    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9469q\" (UniqueName: \"kubernetes.io/projected/85ecc50a-f2d6-4670-aa74-5119b82d7dca-kube-api-access-9469q\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431220    6019 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-secret-alertmanager-main-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431233    6019 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:53.431361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:53.431264    6019 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/85ecc50a-f2d6-4670-aa74-5119b82d7dca-cluster-tls-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:34:54.225438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.225404    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"85ecc50a-f2d6-4670-aa74-5119b82d7dca","Type":"ContainerDied","Data":"311a869fab1de91b7b0b1aa0d07c6315a452fa6e16c4f3eafb1ddc6386f08787"}
Nov 04 02:34:54.225819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.225483    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.225819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.225503    6019 scope.go:122] "RemoveContainer" containerID="1b49299082e4433965739e29d6808e3c0a9de612881c6738d0a440629211dd4a"
Nov 04 02:34:54.235165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.235141    6019 scope.go:122] "RemoveContainer" containerID="e7cca1e604d72ded0ce1c4be348ba56fc8cae6c53ee00e52d7996bfea7923ca5"
Nov 04 02:34:54.244313 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.244295    6019 scope.go:122] "RemoveContainer" containerID="d0f9c795c74091a8cb2b81e67cec4778b24c87266d2856040bdff60a873709da"
Nov 04 02:34:54.250897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.250873    6019 scope.go:122] "RemoveContainer" containerID="355465f32fb82800d4f9ca38adb2c3bc4d95c8ba777cb2d01c93bb4307e99820"
Nov 04 02:34:54.257507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.257484    6019 scope.go:122] "RemoveContainer" containerID="839dbba44c2f83d7ba9dcbd8b6c2539079d3abc2c9a0c69ec76339da2c1e22af"
Nov 04 02:34:54.264441 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.264427    6019 scope.go:122] "RemoveContainer" containerID="379ba94236a643a27bab44298f7088c2bd8e02fd2a1fd23b0552bd21c8684905"
Nov 04 02:34:54.271207 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.271190    6019 scope.go:122] "RemoveContainer" containerID="64153097df5bc324244d7d53581209eda2edab5d59db106e2ed37862bdc76d45"
Nov 04 02:34:54.441402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.441332    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:54.491345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491314    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:54.491607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491589    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:54.491888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491877    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="prom-label-proxy"
Nov 04 02:34:54.491924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491890    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="prom-label-proxy"
Nov 04 02:34:54.491924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491908    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="config-reloader"
Nov 04 02:34:54.491924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491913    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="config-reloader"
Nov 04 02:34:54.491924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491922    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-metric"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491927    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-metric"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491935    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491940    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491946    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="init-config-reloader"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491950    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="init-config-reloader"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491960    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fcf52ec7-7719-406b-8d32-f91721ec6c02" containerName="console"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491964    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcf52ec7-7719-406b-8d32-f91721ec6c02" containerName="console"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491971    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="alertmanager"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491976    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="alertmanager"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491986    6019 cpu_manager.go:414] "containerMap: removing container" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-web"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.491990    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-web"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492039    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-metric"
Nov 04 02:34:54.492043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492047    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492053    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="init-config-reloader"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492059    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcf52ec7-7719-406b-8d32-f91721ec6c02" containerName="console"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492064    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="alertmanager"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492084    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="kube-rbac-proxy-web"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492091    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="prom-label-proxy"
Nov 04 02:34:54.492423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.492098    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" containerName="config-reloader"
Nov 04 02:34:54.495169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.495154    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.541801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541764    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.541935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541804    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.541935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541823    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.541935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541878    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.541935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541927    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-web-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542066 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.541950    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542066 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542023    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frlx\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-kube-api-access-6frlx\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542066 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542041    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542066 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542059    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542114    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542159    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542185    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.542287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.542212    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-out\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.548039 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548019    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 02:34:54.548039 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548019    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 02:34:54.548305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548019    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 02:34:54.548305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548127    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 02:34:54.548305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548230    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 02:34:54.548305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548284    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 02:34:54.548305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548300    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 02:34:54.548556 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548431    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 02:34:54.548680 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.548656    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6g9rb\""
Nov 04 02:34:54.597922 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.597892    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 02:34:54.643034 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.642999    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643042    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643090    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643383    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643518 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643490    6019 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 02:34:54.643518 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643522    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-web-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643567    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643631    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frlx\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-kube-api-access-6frlx\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643657    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643685    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643699    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.643733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643714    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.644187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643752    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.644187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643785    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.644187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643814    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-out\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.644187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.643922    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.645804 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.645751    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7faba029-68fd-4da7-a4da-5cb22dd441bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.647398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.647350    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.647660 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.647576    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-out\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649172 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649146    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649427    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-web-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649499    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649558    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649642    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.649806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.649768    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.650495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.650479    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7faba029-68fd-4da7-a4da-5cb22dd441bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.660424 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.660339    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:54.705049 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.704995    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frlx\" (UniqueName: \"kubernetes.io/projected/7faba029-68fd-4da7-a4da-5cb22dd441bb-kube-api-access-6frlx\") pod \"alertmanager-main-0\" (UID: \"7faba029-68fd-4da7-a4da-5cb22dd441bb\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.810715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.810675    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 02:34:54.978820 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:54.978751    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85ecc50a-f2d6-4670-aa74-5119b82d7dca" path="/var/lib/kubelet/pods/85ecc50a-f2d6-4670-aa74-5119b82d7dca/volumes"
Nov 04 02:34:55.239664 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.239578    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 02:34:55.243384 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:55.243356    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7faba029_68fd_4da7_a4da_5cb22dd441bb.slice/crio-8d7ffbeb5db1bd4b7a2e1afacaee2de9d7aede1d76bfb42f6085c228ef4b345d WatchSource:0}: Error finding container 8d7ffbeb5db1bd4b7a2e1afacaee2de9d7aede1d76bfb42f6085c228ef4b345d: Status 404 returned error can't find the container with id 8d7ffbeb5db1bd4b7a2e1afacaee2de9d7aede1d76bfb42f6085c228ef4b345d
Nov 04 02:34:55.780802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.780752    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8658654fb-59dbg"]
Nov 04 02:34:55.784011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.783987    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.836809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.836758    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 02:34:55.836809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.836779    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 02:34:55.836986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.836852    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 02:34:55.856464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856442    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856474    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmvpg\" (UniqueName: \"kubernetes.io/projected/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-kube-api-access-nmvpg\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856496    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856639 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856573    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-federate-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856682    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856748 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856718    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-serving-certs-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856746    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.856826 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.856808    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-metrics-client-ca\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.886233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.886215    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 02:34:55.886707 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.886688    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jp6w9\""
Nov 04 02:34:55.886851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.886831    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 02:34:55.936577 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.936550    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 02:34:55.937369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.937348    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8658654fb-59dbg"]
Nov 04 02:34:55.957508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957479    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957523    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmvpg\" (UniqueName: \"kubernetes.io/projected/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-kube-api-access-nmvpg\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957556    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957604    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-federate-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957776 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957642    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957776 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957675    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-serving-certs-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957776 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957700    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.957776 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.957727    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-metrics-client-ca\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.958609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.958485    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-serving-certs-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.958807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.958695    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-metrics-client-ca\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.959135 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.959050    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.960254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.960229    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.960353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.960259    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-telemeter-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.960353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.960331    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-federate-client-tls\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:55.960353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:55.960343    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:56.047704 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.047634    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmvpg\" (UniqueName: \"kubernetes.io/projected/eb2814b3-9ba3-4b02-ad23-776c9f286ef3-kube-api-access-nmvpg\") pod \"telemeter-client-8658654fb-59dbg\" (UID: \"eb2814b3-9ba3-4b02-ad23-776c9f286ef3\") " pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:56.097194 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.097171    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg"
Nov 04 02:34:56.234330 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.234296    6019 generic.go:356] "Generic (PLEG): container finished" podID="7faba029-68fd-4da7-a4da-5cb22dd441bb" containerID="9362c4058f462671ce24458e5c62f68ade411a3943263d92567ddc52cd859eaf" exitCode=0
Nov 04 02:34:56.234448 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.234335    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerDied","Data":"9362c4058f462671ce24458e5c62f68ade411a3943263d92567ddc52cd859eaf"}
Nov 04 02:34:56.234448 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.234389    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"8d7ffbeb5db1bd4b7a2e1afacaee2de9d7aede1d76bfb42f6085c228ef4b345d"}
Nov 04 02:34:56.530049 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.530027    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8658654fb-59dbg"]
Nov 04 02:34:56.534028 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:34:56.533405    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2814b3_9ba3_4b02_ad23_776c9f286ef3.slice/crio-a2eaf1c6e48c016e4dead5026004ef16d6b0e780280eae64dc46fc6fa1ec3b30 WatchSource:0}: Error finding container a2eaf1c6e48c016e4dead5026004ef16d6b0e780280eae64dc46fc6fa1ec3b30: Status 404 returned error can't find the container with id a2eaf1c6e48c016e4dead5026004ef16d6b0e780280eae64dc46fc6fa1ec3b30
Nov 04 02:34:56.536117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:56.535638    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:34:57.240933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.240899    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"331123e748bbb6399da987fd8651075706b89acd486e35309692c9d42ec0c2d9"}
Nov 04 02:34:57.241233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.240977    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"24ab90fd4f784baa6fb10e509fa7b97c1bf449bc0154da04d2f7653daf06aad4"}
Nov 04 02:34:57.241233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.241005    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"c32c2aa219a2a05bce08f6b91e0f6325fdc2341c54c34b3da3302a81b46874e3"}
Nov 04 02:34:57.241233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.241042    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"c1f0e13b07981d1ef68b66f09ad56b8b34d928666e61e1e0f851c49c1055a073"}
Nov 04 02:34:57.241233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.241097    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"53fdb34975949fbe9195598cc887a57dce1ec64ba06feeb0510058b86dc8ad0d"}
Nov 04 02:34:57.241233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.241133    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7faba029-68fd-4da7-a4da-5cb22dd441bb","Type":"ContainerStarted","Data":"0c01b0956aaa1fcd26cc007ac867b2469626aae47faa8bca832b26f6b16bc257"}
Nov 04 02:34:57.242207 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.242180    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg" event={"ID":"eb2814b3-9ba3-4b02-ad23-776c9f286ef3","Type":"ContainerStarted","Data":"a2eaf1c6e48c016e4dead5026004ef16d6b0e780280eae64dc46fc6fa1ec3b30"}
Nov 04 02:34:57.369255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:57.369186    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.357398612 podStartE2EDuration="3.369168339s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.769727ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:34:57.368440047 +0000 UTC m=+52.509453479" watchObservedRunningTime="2026-11-04 02:34:57.369168339 +0000 UTC m=+52.510181724"
Nov 04 02:34:59.252059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.252024    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg" event={"ID":"eb2814b3-9ba3-4b02-ad23-776c9f286ef3","Type":"ContainerStarted","Data":"bbfb839791a620483682ad2e368ae2a30964f85433d7c4afa662480f7cf89b2e"}
Nov 04 02:34:59.252455 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.252090    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg" event={"ID":"eb2814b3-9ba3-4b02-ad23-776c9f286ef3","Type":"ContainerStarted","Data":"4d013da66b649478b45849bd282eaae7bb38eb8727c3fdcd2aa0a7d43f1e3981"}
Nov 04 02:34:59.252455 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.252108    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg" event={"ID":"eb2814b3-9ba3-4b02-ad23-776c9f286ef3","Type":"ContainerStarted","Data":"05ea97ccfe5d813b06e0e748489e086f9a06b2386c873d2f56aafd000614af04"}
Nov 04 02:34:59.520767 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.520663    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8658654fb-59dbg" podStartSLOduration=2.175162017 podStartE2EDuration="4.520649671s" totalImagesPullingTime="2.345487654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:34:59.470942229 +0000 UTC m=+54.611955636" watchObservedRunningTime="2026-11-04 02:34:59.520649671 +0000 UTC m=+54.661663076"
Nov 04 02:34:59.908906 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.908837    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:34:59.910782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.910763    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.971382 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.971361    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:34:59.994702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994679    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqp2l\" (UniqueName: \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994706    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994737    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994754    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994913 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994895    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994960 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994929    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.994991 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.994966    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:34:59.995045 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:34:59.995032    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.095833 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095787    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.095833 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095825    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095854    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095885    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095909    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.095937    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096015    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqp2l\" (UniqueName: \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096040    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096289    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096674 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096653    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096693    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.096738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096720    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.097002 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.096982    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.099308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.099262    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.100152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.100131    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.157688 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.157656    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqp2l\" (UniqueName: \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\") pod \"console-6dfb499574-p4bp5\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") " pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.186983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.186892    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:35:00.220924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.220902    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:00.333794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.333763    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:35:00.593803 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.593768    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:35:00.597306 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:35:00.597283    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a7177a_a085_413c_beb8_a9219242fa3b.slice/crio-0649f7860a62d4bf8ac3cb380a7096a02ab40216cbc238d4975dcdbe4cbba693 WatchSource:0}: Error finding container 0649f7860a62d4bf8ac3cb380a7096a02ab40216cbc238d4975dcdbe4cbba693: Status 404 returned error can't find the container with id 0649f7860a62d4bf8ac3cb380a7096a02ab40216cbc238d4975dcdbe4cbba693
Nov 04 02:35:00.609009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.608980    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:35:00.754247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:00.754212    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:35:01.261785 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:01.261740    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb499574-p4bp5" event={"ID":"52a7177a-a085-413c-beb8-a9219242fa3b","Type":"ContainerStarted","Data":"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"}
Nov 04 02:35:01.261994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:01.261818    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb499574-p4bp5" event={"ID":"52a7177a-a085-413c-beb8-a9219242fa3b","Type":"ContainerStarted","Data":"0649f7860a62d4bf8ac3cb380a7096a02ab40216cbc238d4975dcdbe4cbba693"}
Nov 04 02:35:01.380651 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:01.380603    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6dfb499574-p4bp5" podStartSLOduration=2.380588953 podStartE2EDuration="2.380588953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:35:01.380167832 +0000 UTC m=+56.521181239" watchObservedRunningTime="2026-11-04 02:35:01.380588953 +0000 UTC m=+56.521602357"
Nov 04 02:35:06.601610 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:35:06.601577    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:35:10.221284 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:10.221242    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:10.221693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:10.221398    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:10.227429 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:10.227401    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:10.299421 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:10.299394    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:35:10.492054 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:10.491967    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:35:24.601035 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:24.600989    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:35:35.515046 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.514989    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-768b65b77f-9z9tz" podUID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" containerName="console" containerID="cri-o://5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44" gracePeriod=15
Nov 04 02:35:35.765258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.765201    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:35:35.880968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.880939    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.880988    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881016    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881063    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881128    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881163    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881193    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881228    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") pod \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\" (UID: \"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e\") "
Nov 04 02:35:35.881432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881360    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:35:35.881432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881392    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:35:35.881637 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881614    6019 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-trusted-ca-bundle\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.881711 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881643    6019 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.881711 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881657    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:35:35.881711 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881691    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:35:35.881829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.881756    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:35:35.884476 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.884433    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "kube-api-access-w5m6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:35:35.884476 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.884461    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:35:35.884626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.884516    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert" pod "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" (UID: "af01f2a0-9285-43ec-9fa8-8bb739ae6a2e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:35:35.982790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982759    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5m6c\" (UniqueName: \"kubernetes.io/projected/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-kube-api-access-w5m6c\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.982790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982784    6019 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-tmp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.982975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982797    6019 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-oauth-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.982975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982811    6019 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.982975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982824    6019 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-oauth-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:35.982975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:35.982836    6019 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e-console-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:35:36.395923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.395889    6019 generic.go:356] "Generic (PLEG): container finished" podID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" containerID="5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44" exitCode=0
Nov 04 02:35:36.396138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.395946    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768b65b77f-9z9tz"
Nov 04 02:35:36.396138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.395949    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768b65b77f-9z9tz" event={"ID":"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e","Type":"ContainerDied","Data":"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"}
Nov 04 02:35:36.396138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.395997    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768b65b77f-9z9tz" event={"ID":"af01f2a0-9285-43ec-9fa8-8bb739ae6a2e","Type":"ContainerDied","Data":"7f88f1d31e5c94cbd845e77a820b70d2e41c9cd5a3e6c00d7da230c7244944d4"}
Nov 04 02:35:36.396138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.396023    6019 scope.go:122] "RemoveContainer" containerID="5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"
Nov 04 02:35:36.405933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.405916    6019 scope.go:122] "RemoveContainer" containerID="5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"
Nov 04 02:35:36.406425 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:35:36.406387    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44\": container with ID starting with 5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44 not found: ID does not exist" containerID="5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"
Nov 04 02:35:36.406590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.406565    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44"} err="failed to get container status \"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44\": rpc error: code = NotFound desc = could not find container \"5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44\": container with ID starting with 5e802b1c6051b0bf7c28686bd731b42e7d8aaae2405e3a45e11fccca08621e44 not found: ID does not exist"
Nov 04 02:35:36.558974 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.558932    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:35:36.601520 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:35:36.601487    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:35:36.608707 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.608684    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768b65b77f-9z9tz"]
Nov 04 02:35:36.978908 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:35:36.978879    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" path="/var/lib/kubelet/pods/af01f2a0-9285-43ec-9fa8-8bb739ae6a2e/volumes"
Nov 04 02:36:06.602311 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:36:06.602271    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:36:07.800040 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:07.800004    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:08.597059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.597026    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c49c9b56d-vdj78"]
Nov 04 02:36:08.597435 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.597421    6019 cpu_manager.go:414] "containerMap: removing container" podUID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" containerName="console"
Nov 04 02:36:08.597478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.597436    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" containerName="console"
Nov 04 02:36:08.597512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.597498    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="af01f2a0-9285-43ec-9fa8-8bb739ae6a2e" containerName="console"
Nov 04 02:36:08.601862 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.601847    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604579 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604552    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-service-ca\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604612    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-trusted-ca-bundle\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604634    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604653    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5vfb\" (UniqueName: \"kubernetes.io/projected/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-kube-api-access-m5vfb\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604672    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-tmp\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604739    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-oauth-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604791    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.604828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.604804    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-oauth-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705148 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705107    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-oauth-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705169    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-service-ca\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705197    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-trusted-ca-bundle\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705224    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705258    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vfb\" (UniqueName: \"kubernetes.io/projected/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-kube-api-access-m5vfb\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705307    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-tmp\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705356    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-oauth-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705522    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.705882 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705858    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-tmp\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.706001 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.705980    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-oauth-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.706092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.706024    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.706158 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.706121    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-service-ca\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.706238 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.706214    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-trusted-ca-bundle\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.708202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.708185    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-serving-cert\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.708255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.708214    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-console-oauth-config\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.721183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.721152    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c49c9b56d-vdj78"]
Nov 04 02:36:08.766487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.766447    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5vfb\" (UniqueName: \"kubernetes.io/projected/a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3-kube-api-access-m5vfb\") pod \"console-6c49c9b56d-vdj78\" (UID: \"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3\") " pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:08.913437 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:08.913341    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:09.158379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.158346    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:09.284332 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.284302    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c49c9b56d-vdj78"]
Nov 04 02:36:09.288332 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:36:09.288308    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1de0f09_1e36_4085_84a7_1a6a8b5c5ad3.slice/crio-c37773cfbd72c50ab0229025b71ddeffdc1e7d3bfeb038d85e9995dc1904a784 WatchSource:0}: Error finding container c37773cfbd72c50ab0229025b71ddeffdc1e7d3bfeb038d85e9995dc1904a784: Status 404 returned error can't find the container with id c37773cfbd72c50ab0229025b71ddeffdc1e7d3bfeb038d85e9995dc1904a784
Nov 04 02:36:09.302303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.302274    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:09.445607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.445527    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:09.519898 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.519858    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49c9b56d-vdj78" event={"ID":"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3","Type":"ContainerStarted","Data":"53de4e7d76963fcb1ef4091adaef6c871813a642415a0e0b309fde1d104c4361"}
Nov 04 02:36:09.520092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.519915    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49c9b56d-vdj78" event={"ID":"a1de0f09-1e36-4085-84a7-1a6a8b5c5ad3","Type":"ContainerStarted","Data":"c37773cfbd72c50ab0229025b71ddeffdc1e7d3bfeb038d85e9995dc1904a784"}
Nov 04 02:36:09.638662 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:09.638562    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6c49c9b56d-vdj78" podStartSLOduration=1.638541768 podStartE2EDuration="1.638541768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:36:09.637815548 +0000 UTC m=+124.778828956" watchObservedRunningTime="2026-11-04 02:36:09.638541768 +0000 UTC m=+124.779555174"
Nov 04 02:36:18.913840 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:18.913800    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:18.914252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:18.913904    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:18.918533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:18.918509    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:19.559234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:19.559207    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c49c9b56d-vdj78"
Nov 04 02:36:19.727350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:19.727314    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:36:20.428861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:20.428825    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:20.571275 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:20.571243    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:24.954614 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:24.954580    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:33.607853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:33.607820    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:33.752841 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:33.752810    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:36.603526 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:36:36.603487    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:36:44.054293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:44.054261    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:36:44.747354 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:44.747296    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6dfb499574-p4bp5" podUID="52a7177a-a085-413c-beb8-a9219242fa3b" containerName="console" containerID="cri-o://9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898" gracePeriod=15
Nov 04 02:36:45.004201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.004177    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:36:45.113388 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113355    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113433    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113494    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113525    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113569    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\" (UniqueName: \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113608    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113718    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113764    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\") pod \"52a7177a-a085-413c-beb8-a9219242fa3b\" (UID: \"52a7177a-a085-413c-beb8-a9219242fa3b\") "
Nov 04 02:36:45.113819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.113805    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114044    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114054    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114176    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114213    6019 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-console-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114232    6019 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.114278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114261    6019 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-oauth-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.114631 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.114606    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:36:45.115985 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.115957    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:36:45.116126 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.116106    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "kube-api-access-hqp2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:36:45.116373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.116358    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config" pod "52a7177a-a085-413c-beb8-a9219242fa3b" (UID: "52a7177a-a085-413c-beb8-a9219242fa3b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:36:45.215619 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.215579    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqp2l\" (UniqueName: \"kubernetes.io/projected/52a7177a-a085-413c-beb8-a9219242fa3b-kube-api-access-hqp2l\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.215619 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.215612    6019 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a7177a-a085-413c-beb8-a9219242fa3b-tmp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.215619 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.215622    6019 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a7177a-a085-413c-beb8-a9219242fa3b-trusted-ca-bundle\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.215857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.215633    6019 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-oauth-config\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.215857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.215643    6019 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52a7177a-a085-413c-beb8-a9219242fa3b-console-serving-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:36:45.661390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.661354    6019 generic.go:356] "Generic (PLEG): container finished" podID="52a7177a-a085-413c-beb8-a9219242fa3b" containerID="9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898" exitCode=0
Nov 04 02:36:45.661587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.661408    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb499574-p4bp5" event={"ID":"52a7177a-a085-413c-beb8-a9219242fa3b","Type":"ContainerDied","Data":"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"}
Nov 04 02:36:45.661587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.661411    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb499574-p4bp5"
Nov 04 02:36:45.661587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.661451    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb499574-p4bp5" event={"ID":"52a7177a-a085-413c-beb8-a9219242fa3b","Type":"ContainerDied","Data":"0649f7860a62d4bf8ac3cb380a7096a02ab40216cbc238d4975dcdbe4cbba693"}
Nov 04 02:36:45.661587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.661470    6019 scope.go:122] "RemoveContainer" containerID="9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"
Nov 04 02:36:45.669938 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.669919    6019 scope.go:122] "RemoveContainer" containerID="9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"
Nov 04 02:36:45.670237 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:36:45.670217    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898\": container with ID starting with 9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898 not found: ID does not exist" containerID="9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"
Nov 04 02:36:45.670335 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.670244    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898"} err="failed to get container status \"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898\": rpc error: code = NotFound desc = could not find container \"9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898\": container with ID starting with 9afe6ed11f3ed349e85d7d0dac3f0c5422d5f05712a951c74a8cfb09b60db898 not found: ID does not exist"
Nov 04 02:36:45.824972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.824937    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:36:45.874864 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:45.874835    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfb499574-p4bp5"]
Nov 04 02:36:46.979473 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:36:46.979441    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52a7177a-a085-413c-beb8-a9219242fa3b" path="/var/lib/kubelet/pods/52a7177a-a085-413c-beb8-a9219242fa3b/volumes"
Nov 04 02:37:06.601880 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:37:06.601842    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:37:17.831814 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:17.831757    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:18.014426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.014389    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:18.208003 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.207924    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:18.901164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.901133    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"]
Nov 04 02:37:18.901533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.901489    6019 cpu_manager.go:414] "containerMap: removing container" podUID="52a7177a-a085-413c-beb8-a9219242fa3b" containerName="console"
Nov 04 02:37:18.901533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.901502    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52a7177a-a085-413c-beb8-a9219242fa3b" containerName="console"
Nov 04 02:37:18.901601 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.901572    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="52a7177a-a085-413c-beb8-a9219242fa3b" containerName="console"
Nov 04 02:37:18.904550 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.904526    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:18.964191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.964161    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pf4\" (UniqueName: \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:18.964299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.964205    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:18.964299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:18.964240    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.007084 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.007037    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 02:37:19.007208 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.007108    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-446w6\""
Nov 04 02:37:19.007257 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.007205    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 02:37:19.007998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.007984    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"]
Nov 04 02:37:19.065198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.065170    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.065334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.065207    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.065334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.065255    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pf4\" (UniqueName: \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.065539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.065523    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.065595 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.065577    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.099918 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.099896    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"]
Nov 04 02:37:19.103309 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.103295    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.125783 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.125755    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pf4\" (UniqueName: \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.161380 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.161318    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"]
Nov 04 02:37:19.166191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.166169    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98zdm\" (UniqueName: \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.166285 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.166199    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.166375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.166349    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.212606 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.212578    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-sxdnd\""
Nov 04 02:37:19.219689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.219666    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:19.267210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.267179    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.267357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.267259    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98zdm\" (UniqueName: \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.267357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.267298    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.267901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.267875    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.268091 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.268051    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.328568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.328540    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98zdm\" (UniqueName: \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.365713 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.365688    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:37:19.370103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.370063    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.384891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.384869    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:19.419054 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.419011    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:19.428000 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.427969    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 02:37:19.428149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.428114    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 02:37:19.428606 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.428586    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 02:37:19.431724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.431650    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 02:37:19.432380 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.432332    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 02:37:19.432380 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.432379    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 02:37:19.432901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.432881    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n69rv\""
Nov 04 02:37:19.433037 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.433019    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468523    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468622    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468656    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8l6q\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468699    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468735    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468761    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.468850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.468802    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.529392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.529364    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:19.569709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569682    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569710    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8l6q\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569735    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569756    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569772    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569993 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569892    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.569993 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.569940    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.570366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.570346    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.572260 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.572230    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.572350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.572286    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.572395 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.572354    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.572395 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.572371    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.630243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.630223    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.630507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.630491    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8l6q\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\") pod \"istiod-openshift-gateway-7876bfcb74-wm9hl\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.637442 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.637420    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"]
Nov 04 02:37:19.641184 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:37:19.641154    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34325451_132d_48b6_bd23_1ec4da0d6c99.slice/crio-128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4 WatchSource:0}: Error finding container 128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4: Status 404 returned error can't find the container with id 128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4
Nov 04 02:37:19.681844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.681789    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:19.786607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.786574    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx" event={"ID":"34325451-132d-48b6-bd23-1ec4da0d6c99","Type":"ContainerStarted","Data":"128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4"}
Nov 04 02:37:19.790544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:19.790517    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"]
Nov 04 02:37:19.797555 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:37:19.797532    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be82c96_ccd4_4425_8189_645af41d0af2.slice/crio-3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3 WatchSource:0}: Error finding container 3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3: Status 404 returned error can't find the container with id 3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3
Nov 04 02:37:20.056024 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:20.055999    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:37:20.059023 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:37:20.058995    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67122984_cccf_4f01_b0ca_64d7ebd05c70.slice/crio-b4d4e20043730c1dec3c512ae1234947bbb76578b87f95590919122c8e084fe7 WatchSource:0}: Error finding container b4d4e20043730c1dec3c512ae1234947bbb76578b87f95590919122c8e084fe7: Status 404 returned error can't find the container with id b4d4e20043730c1dec3c512ae1234947bbb76578b87f95590919122c8e084fe7
Nov 04 02:37:20.301946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:20.301908    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:20.662214 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:20.662179    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:20.791730 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:20.791695    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" event={"ID":"67122984-cccf-4f01-b0ca-64d7ebd05c70","Type":"ContainerStarted","Data":"b4d4e20043730c1dec3c512ae1234947bbb76578b87f95590919122c8e084fe7"}
Nov 04 02:37:20.792778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:20.792749    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerStarted","Data":"3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3"}
Nov 04 02:37:24.599398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:24.599361    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:37:24.599735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:24.599436    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:37:24.818528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:24.818493    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" event={"ID":"67122984-cccf-4f01-b0ca-64d7ebd05c70","Type":"ContainerStarted","Data":"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"}
Nov 04 02:37:24.818723 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:24.818596    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:25.040786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:25.040721    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" podStartSLOduration=1.503325866 podStartE2EDuration="6.040700994s" totalImagesPullingTime="4.537375128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:37:24.989896477 +0000 UTC m=+200.130909884" watchObservedRunningTime="2026-11-04 02:37:25.040700994 +0000 UTC m=+200.181714404"
Nov 04 02:37:25.311658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:25.311575    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:25.825815 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:25.825488    6019 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-wm9hl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:37:25.825815 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:25.825547    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:37:26.040698 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:26.040602    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:26.184842 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:26.184746    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:27.485948 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.485902    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:27.646766 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.646739    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:27.804371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.804345    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:27.832904 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.832872    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerStarted","Data":"38849b6c8858a1b2fc0e3d5cffa295c1102452111df427b388c44ebc4d963015"}
Nov 04 02:37:27.834746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.834711    6019 generic.go:356] "Generic (PLEG): container finished" podID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerID="f5599908e4c92292c00a0748eecda6369d34f5eaed3c64db61d645d03d31899f" exitCode=0
Nov 04 02:37:27.834845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.834767    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx" event={"ID":"34325451-132d-48b6-bd23-1ec4da0d6c99","Type":"ContainerDied","Data":"f5599908e4c92292c00a0748eecda6369d34f5eaed3c64db61d645d03d31899f"}
Nov 04 02:37:27.953440 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:27.953408    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:28.824299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:28.824270    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:37:28.840194 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:28.840165    6019 generic.go:356] "Generic (PLEG): container finished" podID="6be82c96-ccd4-4425-8189-645af41d0af2" containerID="38849b6c8858a1b2fc0e3d5cffa295c1102452111df427b388c44ebc4d963015" exitCode=0
Nov 04 02:37:28.840320 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:28.840231    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerDied","Data":"38849b6c8858a1b2fc0e3d5cffa295c1102452111df427b388c44ebc4d963015"}
Nov 04 02:37:30.851136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:30.851102    6019 generic.go:356] "Generic (PLEG): container finished" podID="6be82c96-ccd4-4425-8189-645af41d0af2" containerID="fe6a79c47615e7f32bc890a075aad4089eaf3ad747aeb01f3fb72bfdf0578e9e" exitCode=0
Nov 04 02:37:30.851576 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:30.851171    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerDied","Data":"fe6a79c47615e7f32bc890a075aad4089eaf3ad747aeb01f3fb72bfdf0578e9e"}
Nov 04 02:37:31.662566 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:31.662530    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:37:32.862605 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:32.862567    6019 generic.go:356] "Generic (PLEG): container finished" podID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerID="cce7b4c7b1071a572ea10d015fb780185e85d654279d64e4576079d05b321225" exitCode=0
Nov 04 02:37:32.863048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:32.862659    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx" event={"ID":"34325451-132d-48b6-bd23-1ec4da0d6c99","Type":"ContainerDied","Data":"cce7b4c7b1071a572ea10d015fb780185e85d654279d64e4576079d05b321225"}
Nov 04 02:37:36.611967 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:37:36.611900    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:37:38.894111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:38.894042    6019 generic.go:356] "Generic (PLEG): container finished" podID="6be82c96-ccd4-4425-8189-645af41d0af2" containerID="dda4db4ea1a3626d96ae1b2aab765a9fee1a688dae82cd0be19e27b2fcdf0737" exitCode=0
Nov 04 02:37:38.894586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:38.894141    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerDied","Data":"dda4db4ea1a3626d96ae1b2aab765a9fee1a688dae82cd0be19e27b2fcdf0737"}
Nov 04 02:37:40.026541 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.026518    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:40.157094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.157003    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\") pod \"6be82c96-ccd4-4425-8189-645af41d0af2\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") "
Nov 04 02:37:40.157094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.157061    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\" (UniqueName: \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\") pod \"6be82c96-ccd4-4425-8189-645af41d0af2\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") "
Nov 04 02:37:40.157250 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.157190    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\") pod \"6be82c96-ccd4-4425-8189-645af41d0af2\" (UID: \"6be82c96-ccd4-4425-8189-645af41d0af2\") "
Nov 04 02:37:40.157778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.157734    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle" pod "6be82c96-ccd4-4425-8189-645af41d0af2" (UID: "6be82c96-ccd4-4425-8189-645af41d0af2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:37:40.162274 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.162248    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util" pod "6be82c96-ccd4-4425-8189-645af41d0af2" (UID: "6be82c96-ccd4-4425-8189-645af41d0af2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:37:40.162718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.162693    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm" pod "6be82c96-ccd4-4425-8189-645af41d0af2" (UID: "6be82c96-ccd4-4425-8189-645af41d0af2"). InnerVolumeSpecName "kube-api-access-98zdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:37:40.257939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.257911    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98zdm\" (UniqueName: \"kubernetes.io/projected/6be82c96-ccd4-4425-8189-645af41d0af2-kube-api-access-98zdm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:40.257939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.257935    6019 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-util\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:40.257939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.257944    6019 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6be82c96-ccd4-4425-8189-645af41d0af2-bundle\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:40.904019 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.903986    6019 generic.go:356] "Generic (PLEG): container finished" podID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerID="c2eaccac0a8e7914069a1b0dc8f3cef4c80e40d80d19f9e6e6c70c6e09d5bf57" exitCode=0
Nov 04 02:37:40.904203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.904081    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx" event={"ID":"34325451-132d-48b6-bd23-1ec4da0d6c99","Type":"ContainerDied","Data":"c2eaccac0a8e7914069a1b0dc8f3cef4c80e40d80d19f9e6e6c70c6e09d5bf57"}
Nov 04 02:37:40.905701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.905674    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9" event={"ID":"6be82c96-ccd4-4425-8189-645af41d0af2","Type":"ContainerDied","Data":"3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3"}
Nov 04 02:37:40.905701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.905697    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xwtv9"
Nov 04 02:37:40.905846 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:40.905718    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c49849f478b1243a96a3f181c1ea225c4b817f7bb3cddc25a1774b0e358c3c3"
Nov 04 02:37:42.033879 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.033858    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:42.172017 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.171948    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\") pod \"34325451-132d-48b6-bd23-1ec4da0d6c99\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") "
Nov 04 02:37:42.172017 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.171997    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\") pod \"34325451-132d-48b6-bd23-1ec4da0d6c99\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") "
Nov 04 02:37:42.172249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.172042    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\" (UniqueName: \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\") pod \"34325451-132d-48b6-bd23-1ec4da0d6c99\" (UID: \"34325451-132d-48b6-bd23-1ec4da0d6c99\") "
Nov 04 02:37:42.172315 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.172292    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle" pod "34325451-132d-48b6-bd23-1ec4da0d6c99" (UID: "34325451-132d-48b6-bd23-1ec4da0d6c99"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:37:42.174327 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.174298    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4" pod "34325451-132d-48b6-bd23-1ec4da0d6c99" (UID: "34325451-132d-48b6-bd23-1ec4da0d6c99"). InnerVolumeSpecName "kube-api-access-j6pf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:37:42.179426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.179401    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util" pod "34325451-132d-48b6-bd23-1ec4da0d6c99" (UID: "34325451-132d-48b6-bd23-1ec4da0d6c99"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:37:42.273566 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.273542    6019 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-bundle\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:42.273566 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.273564    6019 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34325451-132d-48b6-bd23-1ec4da0d6c99-util\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:42.273713 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.273574    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6pf4\" (UniqueName: \"kubernetes.io/projected/34325451-132d-48b6-bd23-1ec4da0d6c99-kube-api-access-j6pf4\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:37:42.915055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.915020    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx" event={"ID":"34325451-132d-48b6-bd23-1ec4da0d6c99","Type":"ContainerDied","Data":"128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4"}
Nov 04 02:37:42.915233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.915099    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="128a04053a2c8a2975b5c7ed11c3cc52fb19815b1377f6cd3ad0d318933826f4"
Nov 04 02:37:42.915233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:42.915032    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644d48lx"
Nov 04 02:37:46.242173 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:37:46.242143    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:06.601983 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:38:06.601943    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:38:11.059202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:11.059160    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:11.551323 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:11.551286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:12.132942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.132911    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:12.341089 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341048    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"]
Nov 04 02:38:12.341377 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341365    6019 cpu_manager.go:414] "containerMap: removing container" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="extract"
Nov 04 02:38:12.341418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341379    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="extract"
Nov 04 02:38:12.341418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341395    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="util"
Nov 04 02:38:12.341418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341401    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="util"
Nov 04 02:38:12.341418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341411    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="extract"
Nov 04 02:38:12.341418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341416    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="extract"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341429    6019 cpu_manager.go:414] "containerMap: removing container" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="util"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341434    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="util"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341444    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="pull"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341448    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="pull"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341462    6019 cpu_manager.go:414] "containerMap: removing container" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="pull"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341466    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="pull"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341513    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="34325451-132d-48b6-bd23-1ec4da0d6c99" containerName="extract"
Nov 04 02:38:12.341563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.341521    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="6be82c96-ccd4-4425-8189-645af41d0af2" containerName="extract"
Nov 04 02:38:12.343477 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.343462    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.405809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405742    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9v8d\" (UniqueName: \"kubernetes.io/projected/9f43262e-a53f-4e71-9093-5f5975e06e22-kube-api-access-x9v8d\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.405942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405834    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-tmp\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.405942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405912    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.406011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405955    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.406011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405976    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-manifests\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.406011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.405998    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.419803 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.419779    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:12.470089 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470056    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 02:38:12.470222 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470061    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 02:38:12.470222 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470118    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 02:38:12.470368 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470351    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-8qrsw\""
Nov 04 02:38:12.470511 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470498    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 02:38:12.470568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.470535    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 02:38:12.506656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506626    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9v8d\" (UniqueName: \"kubernetes.io/projected/9f43262e-a53f-4e71-9093-5f5975e06e22-kube-api-access-x9v8d\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.506794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506668    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-tmp\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.506794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506717    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.506794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506767    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.506954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506837    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-manifests\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.506954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.506881    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.507247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.507220    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-manifests\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.509476 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.509452    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.509617 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.509601    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.509729 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.509706    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f43262e-a53f-4e71-9093-5f5975e06e22-tmp\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.509960 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.509941    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f43262e-a53f-4e71-9093-5f5975e06e22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.587812 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.587777    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9v8d\" (UniqueName: \"kubernetes.io/projected/9f43262e-a53f-4e71-9093-5f5975e06e22-kube-api-access-x9v8d\") pod \"opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd\" (UID: \"9f43262e-a53f-4e71-9093-5f5975e06e22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.608796 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.608765    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"]
Nov 04 02:38:12.610995 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.610942    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:12.655984 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.655903    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:12.763287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.763231    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:12.911296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:12.911220    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:13.141127 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:13.141095    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"]
Nov 04 02:38:13.144094 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:38:13.144046    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f43262e_a53f_4e71_9093_5f5975e06e22.slice/crio-d4e5dec44738bcefb2e844875c938ddd611bc9e363559037a9d650260e82a359 WatchSource:0}: Error finding container d4e5dec44738bcefb2e844875c938ddd611bc9e363559037a9d650260e82a359: Status 404 returned error can't find the container with id d4e5dec44738bcefb2e844875c938ddd611bc9e363559037a9d650260e82a359
Nov 04 02:38:14.036335 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:14.036297    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd" event={"ID":"9f43262e-a53f-4e71-9093-5f5975e06e22","Type":"ContainerStarted","Data":"d4e5dec44738bcefb2e844875c938ddd611bc9e363559037a9d650260e82a359"}
Nov 04 02:38:15.067758 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:15.067682    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:15.474111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:15.474005    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:17.057998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:17.057966    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd" event={"ID":"9f43262e-a53f-4e71-9093-5f5975e06e22","Type":"ContainerStarted","Data":"761f926ce98f5e07d78ef5cde9bb21bc37dc10067d18d4d37a563cc4442f464e"}
Nov 04 02:38:18.063335 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:18.063298    6019 generic.go:356] "Generic (PLEG): container finished" podID="9f43262e-a53f-4e71-9093-5f5975e06e22" containerID="761f926ce98f5e07d78ef5cde9bb21bc37dc10067d18d4d37a563cc4442f464e" exitCode=0
Nov 04 02:38:18.063732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:18.063374    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd" event={"ID":"9f43262e-a53f-4e71-9093-5f5975e06e22","Type":"ContainerDied","Data":"761f926ce98f5e07d78ef5cde9bb21bc37dc10067d18d4d37a563cc4442f464e"}
Nov 04 02:38:19.069369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:19.069272    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd" event={"ID":"9f43262e-a53f-4e71-9093-5f5975e06e22","Type":"ContainerStarted","Data":"e1ff0478be3c642cf43bb7ad28bc39dbbd20ecb58d12cd751cf10543a5a026e3"}
Nov 04 02:38:19.069369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:19.069355    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:19.312430 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:19.312376    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd" podStartSLOduration=2.726485768 podStartE2EDuration="7.312363087s" totalImagesPullingTime="4.176795953s" totalInitContainerRuntime="409.081366ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:38:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:38:19.263260149 +0000 UTC m=+254.404273557" watchObservedRunningTime="2026-11-04 02:38:19.312363087 +0000 UTC m=+254.453376494"
Nov 04 02:38:25.050569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:25.050535    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:25.223501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:25.223470    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:25.392529 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:25.392450    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:25.691444 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:25.691353    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:25.855811 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:25.855769    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:26.929009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:26.928961    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:27.968113 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:27.968086    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:28.704366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:28.704330    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:29.246683 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:29.246655    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:29.671147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:29.671112    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:29.863899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:29.863868    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:30.075426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:30.075397    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bfc68bc9f-xrnqd"
Nov 04 02:38:30.245835 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:30.245803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:31.240993 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:31.240964    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:33.795292 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:33.795256    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:36.495565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:36.495534    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:36.602599 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:38:36.602561    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:38:37.191299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:37.191269    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:37.344125 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:37.344063    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:38.025228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:38.025193    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:41.125609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:41.125574    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:41.325063 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:41.325017    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:41.927968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:41.927924    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:42.285477 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:42.285437    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:42.430742 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:42.430702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:42.804732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:42.804687    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:42.988392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:42.988358    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:43.124904 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.124828    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"]
Nov 04 02:38:43.128178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.128161    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.149424 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.149376    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:43.186175 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.186144    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"]
Nov 04 02:38:43.229897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.229861    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 02:38:43.229897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.229874    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 02:38:43.230123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.229904    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9vcnt\""
Nov 04 02:38:43.263958 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.263928    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvqv6\" (UniqueName: \"kubernetes.io/projected/189d27e4-0565-4c01-836a-ed99a907f34c-kube-api-access-xvqv6\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.264142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.263993    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189d27e4-0565-4c01-836a-ed99a907f34c-tmp\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.264142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.264054    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189d27e4-0565-4c01-836a-ed99a907f34c-tls-certs\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.327207 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.327151    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:43.364987 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.364957    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvqv6\" (UniqueName: \"kubernetes.io/projected/189d27e4-0565-4c01-836a-ed99a907f34c-kube-api-access-xvqv6\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.365229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.365018    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189d27e4-0565-4c01-836a-ed99a907f34c-tmp\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.365229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.365053    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189d27e4-0565-4c01-836a-ed99a907f34c-tls-certs\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.367617 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.367590    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189d27e4-0565-4c01-836a-ed99a907f34c-tmp\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.367794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.367776    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189d27e4-0565-4c01-836a-ed99a907f34c-tls-certs\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.425922 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.425848    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvqv6\" (UniqueName: \"kubernetes.io/projected/189d27e4-0565-4c01-836a-ed99a907f34c-kube-api-access-xvqv6\") pod \"kube-auth-proxy-54497bf445-q5p8t\" (UID: \"189d27e4-0565-4c01-836a-ed99a907f34c\") " pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.439620 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.439591    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"
Nov 04 02:38:43.709175 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.709084    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:43.818527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:43.818495    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54497bf445-q5p8t"]
Nov 04 02:38:43.821965 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:38:43.821926    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189d27e4_0565_4c01_836a_ed99a907f34c.slice/crio-245ddb8d215bf480c388c2a7faeb26e5f559902a3918c108731977f018efe0ce WatchSource:0}: Error finding container 245ddb8d215bf480c388c2a7faeb26e5f559902a3918c108731977f018efe0ce: Status 404 returned error can't find the container with id 245ddb8d215bf480c388c2a7faeb26e5f559902a3918c108731977f018efe0ce
Nov 04 02:38:44.022471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:44.022433    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:44.172926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:44.172892    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t" event={"ID":"189d27e4-0565-4c01-836a-ed99a907f34c","Type":"ContainerStarted","Data":"245ddb8d215bf480c388c2a7faeb26e5f559902a3918c108731977f018efe0ce"}
Nov 04 02:38:44.186785 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:44.186753    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:45.395607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:45.395566    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:46.383227 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:46.383193    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:46.534496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:46.534466    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:47.440699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:47.440643    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:47.592205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:47.592168    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:48.192972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:48.192943    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t" event={"ID":"189d27e4-0565-4c01-836a-ed99a907f34c","Type":"ContainerStarted","Data":"98343f0c1bfa25ca314fa49220a7d3e472e8836e8ee37465dc5eeeb59f528b96"}
Nov 04 02:38:48.308998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:48.308942    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54497bf445-q5p8t" podStartSLOduration=1.159397633 podStartE2EDuration="5.308928185s" totalImagesPullingTime="4.149530552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:38:48.308426443 +0000 UTC m=+283.449439851" watchObservedRunningTime="2026-11-04 02:38:48.308928185 +0000 UTC m=+283.449941653"
Nov 04 02:38:51.120656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:51.120618    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:53.043128 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:53.043053    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:53.186586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:53.186554    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:53.644946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:53.644911    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:53.790626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:53.790587    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:54.041544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:54.041506    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:54.192746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:54.192714    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:59.656538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.656500    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:38:59.743439 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.743399    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:38:59.748012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.747997    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:38:59.802957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.802785    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 02:38:59.852375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.852352    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4fz9l\""
Nov 04 02:38:59.852693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.852531    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 02:38:59.852693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.852583    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 02:38:59.853554 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.853539    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:38:59.909379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.909316    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9hx\" (UniqueName: \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:38:59.909379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.909349    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:38:59.989306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:38:59.989280    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:39:00.010169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.010129    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9hx\" (UniqueName: \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:00.010327 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.010174    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:00.010730 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.010681    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:00.010881 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.010824    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:00.072653 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.072626    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9hx\" (UniqueName: \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\") pod \"limitador-limitador-74dfcfb894-hxnmh\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:00.152909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.152877    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:00.359483 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.359457    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:00.728690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:00.728619    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:39:00.731883 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:39:00.731858    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df976bf_7dfc_4ee4_b51a_816a8bd066b6.slice/crio-5a96d8c63878c2072a4657e4beffc723a0e9e477e2f99750644538e818864da4 WatchSource:0}: Error finding container 5a96d8c63878c2072a4657e4beffc723a0e9e477e2f99750644538e818864da4: Status 404 returned error can't find the container with id 5a96d8c63878c2072a4657e4beffc723a0e9e477e2f99750644538e818864da4
Nov 04 02:39:01.248629 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:01.248584    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" event={"ID":"5df976bf-7dfc-4ee4-b51a-816a8bd066b6","Type":"ContainerStarted","Data":"5a96d8c63878c2072a4657e4beffc723a0e9e477e2f99750644538e818864da4"}
Nov 04 02:39:01.450553 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:01.450517    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:01.612333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:01.612255    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:02.069619 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:02.069586    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:02.214984 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:02.214946    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:04.055972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.055943    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:39:04.062707 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.062680    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:39:04.115100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.115055    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:39:04.142482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.142456    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpqd5\" (UniqueName: \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\") pod \"authorino-56c6c9ddd5-rxglf\" (UID: \"474f04ff-08a1-476b-8a8b-a81a6160d185\") " pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:39:04.162365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.162332    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wpmvk\""
Nov 04 02:39:04.243246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.243207    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpqd5\" (UniqueName: \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\") pod \"authorino-56c6c9ddd5-rxglf\" (UID: \"474f04ff-08a1-476b-8a8b-a81a6160d185\") " pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:39:04.305736 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.305705    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpqd5\" (UniqueName: \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\") pod \"authorino-56c6c9ddd5-rxglf\" (UID: \"474f04ff-08a1-476b-8a8b-a81a6160d185\") " pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:39:04.391658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.391637    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:39:04.761178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:04.761148    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:39:04.764284 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:39:04.764253    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod474f04ff_08a1_476b_8a8b_a81a6160d185.slice/crio-65e230cc689e4e094e784c856d4ed99996203cbb3144ef2310318ace774cf268 WatchSource:0}: Error finding container 65e230cc689e4e094e784c856d4ed99996203cbb3144ef2310318ace774cf268: Status 404 returned error can't find the container with id 65e230cc689e4e094e784c856d4ed99996203cbb3144ef2310318ace774cf268
Nov 04 02:39:05.008037 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.008007    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:39:05.008945 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.008927    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:39:05.055590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.055531    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:05.199771 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.199741    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:05.267337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.267301    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" event={"ID":"474f04ff-08a1-476b-8a8b-a81a6160d185","Type":"ContainerStarted","Data":"65e230cc689e4e094e784c856d4ed99996203cbb3144ef2310318ace774cf268"}
Nov 04 02:39:05.268721 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.268684    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" event={"ID":"5df976bf-7dfc-4ee4-b51a-816a8bd066b6","Type":"ContainerStarted","Data":"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"}
Nov 04 02:39:05.268861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.268816    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:05.384197 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:05.384111    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" podStartSLOduration=2.734303263 podStartE2EDuration="6.384098877s" totalImagesPullingTime="3.649795614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:39:05.383726976 +0000 UTC m=+300.524740380" watchObservedRunningTime="2026-11-04 02:39:05.384098877 +0000 UTC m=+300.525112282"
Nov 04 02:39:06.605374 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:39:06.605335    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:39:07.279672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:07.279634    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" event={"ID":"474f04ff-08a1-476b-8a8b-a81a6160d185","Type":"ContainerStarted","Data":"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"}
Nov 04 02:39:07.394023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:07.393926    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" podStartSLOduration=1.102895811 podStartE2EDuration="3.393912651s" totalImagesPullingTime="2.29101684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:39:07.393344138 +0000 UTC m=+302.534357544" watchObservedRunningTime="2026-11-04 02:39:07.393912651 +0000 UTC m=+302.534926059"
Nov 04 02:39:07.654838 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:07.654748    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:07.799851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:07.799821    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:13.053230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:13.053192    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:13.200896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:13.200861    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:13.345488 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:13.345408    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:13.491415 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:13.491382    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:13.863980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:13.863945    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:14.035133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:14.035036    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:16.275223 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:16.275184    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:19.285918 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.285888    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:39:19.286431 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.286166    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" podUID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" containerName="limitador" containerID="cri-o://092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b" gracePeriod=30
Nov 04 02:39:19.837463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.837437    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:19.880371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.880281    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\" (UniqueName: \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\") pod \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") "
Nov 04 02:39:19.880371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.880327    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\" (UniqueName: \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\") pod \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\" (UID: \"5df976bf-7dfc-4ee4-b51a-816a8bd066b6\") "
Nov 04 02:39:19.880686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.880665    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file" pod "5df976bf-7dfc-4ee4-b51a-816a8bd066b6" (UID: "5df976bf-7dfc-4ee4-b51a-816a8bd066b6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:39:19.882843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.882815    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx" pod "5df976bf-7dfc-4ee4-b51a-816a8bd066b6" (UID: "5df976bf-7dfc-4ee4-b51a-816a8bd066b6"). InnerVolumeSpecName "kube-api-access-np9hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:39:19.981202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.981169    6019 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-config-file\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:39:19.981202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:19.981196    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np9hx\" (UniqueName: \"kubernetes.io/projected/5df976bf-7dfc-4ee4-b51a-816a8bd066b6-kube-api-access-np9hx\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:39:20.344567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.344514    6019 generic.go:356] "Generic (PLEG): container finished" podID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" containerID="092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b" exitCode=0
Nov 04 02:39:20.345123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.344589    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" event={"ID":"5df976bf-7dfc-4ee4-b51a-816a8bd066b6","Type":"ContainerDied","Data":"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"}
Nov 04 02:39:20.345123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.344596    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"
Nov 04 02:39:20.345123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.344708    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hxnmh" event={"ID":"5df976bf-7dfc-4ee4-b51a-816a8bd066b6","Type":"ContainerDied","Data":"5a96d8c63878c2072a4657e4beffc723a0e9e477e2f99750644538e818864da4"}
Nov 04 02:39:20.345123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.344741    6019 scope.go:122] "RemoveContainer" containerID="092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"
Nov 04 02:39:20.353183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.353166    6019 scope.go:122] "RemoveContainer" containerID="092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"
Nov 04 02:39:20.353460 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:39:20.353438    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b\": container with ID starting with 092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b not found: ID does not exist" containerID="092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"
Nov 04 02:39:20.353529 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.353467    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b"} err="failed to get container status \"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b\": rpc error: code = NotFound desc = could not find container \"092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b\": container with ID starting with 092adcfc74a60c6f573c304823decf2681248b8355864c4e549b43c363679f4b not found: ID does not exist"
Nov 04 02:39:20.508026 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.507987    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:39:20.512273 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.512245    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hxnmh"]
Nov 04 02:39:20.980227 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:20.980199    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" path="/var/lib/kubelet/pods/5df976bf-7dfc-4ee4-b51a-816a8bd066b6/volumes"
Nov 04 02:39:21.928597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:21.928566    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:22.071012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:22.070982    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:24.875175 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:24.875142    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:25.024620 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:25.024587    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:26.027510 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:26.027447    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:35.773090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.773039    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:35.795323 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.795297    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"]
Nov 04 02:39:35.795628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.795617    6019 cpu_manager.go:414] "containerMap: removing container" podUID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" containerName="limitador"
Nov 04 02:39:35.795673 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.795630    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" containerName="limitador"
Nov 04 02:39:35.795718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.795710    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="5df976bf-7dfc-4ee4-b51a-816a8bd066b6" containerName="limitador"
Nov 04 02:39:35.800357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.800337    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:35.904837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.904804    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 04 02:39:35.904837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.904829    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 04 02:39:35.905086 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.904842    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4fz9l\""
Nov 04 02:39:35.905980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.905954    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"]
Nov 04 02:39:35.917029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.916995    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sflt9\" (UniqueName: \"kubernetes.io/projected/c325cb7f-d5e1-4efd-84db-0dfec36c5878-kube-api-access-sflt9\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:35.917187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.917058    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c325cb7f-d5e1-4efd-84db-0dfec36c5878-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:35.917267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.917209    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c325cb7f-d5e1-4efd-84db-0dfec36c5878-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:35.964219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:35.964186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:36.018587 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.018562    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c325cb7f-d5e1-4efd-84db-0dfec36c5878-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.018688 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.018614    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sflt9\" (UniqueName: \"kubernetes.io/projected/c325cb7f-d5e1-4efd-84db-0dfec36c5878-kube-api-access-sflt9\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.018688 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.018663    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c325cb7f-d5e1-4efd-84db-0dfec36c5878-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.019386 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.019328    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c325cb7f-d5e1-4efd-84db-0dfec36c5878-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.021161 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.021143    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c325cb7f-d5e1-4efd-84db-0dfec36c5878-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.079458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.079400    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sflt9\" (UniqueName: \"kubernetes.io/projected/c325cb7f-d5e1-4efd-84db-0dfec36c5878-kube-api-access-sflt9\") pod \"kuadrant-console-plugin-66c4fb575d-48kqg\" (UID: \"c325cb7f-d5e1-4efd-84db-0dfec36c5878\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.111252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.111231    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"
Nov 04 02:39:36.480718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:36.480649    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg"]
Nov 04 02:39:36.484021 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:39:36.483992    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc325cb7f_d5e1_4efd_84db_0dfec36c5878.slice/crio-845ab8523f9227c266683f6264f507a63b9d341c4789ae6e0aa4944eb09958ab WatchSource:0}: Error finding container 845ab8523f9227c266683f6264f507a63b9d341c4789ae6e0aa4944eb09958ab: Status 404 returned error can't find the container with id 845ab8523f9227c266683f6264f507a63b9d341c4789ae6e0aa4944eb09958ab
Nov 04 02:39:36.601641 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:39:36.601606    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:39:37.418431 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:37.418399    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg" event={"ID":"c325cb7f-d5e1-4efd-84db-0dfec36c5878","Type":"ContainerStarted","Data":"845ab8523f9227c266683f6264f507a63b9d341c4789ae6e0aa4944eb09958ab"}
Nov 04 02:39:39.693533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:39.693502    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:39.839210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:39.839179    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:40.375898 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.375862    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"]
Nov 04 02:39:40.379953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.379922    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.433000 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.432967    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:39:40.457555 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.457524    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5lhj\" (UniqueName: \"kubernetes.io/projected/fadfa4ac-e093-4930-9750-0bca1446d2bf-kube-api-access-h5lhj\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.457732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.457584    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fadfa4ac-e093-4930-9750-0bca1446d2bf-data\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.467763 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.467730    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:39:40.482015 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.481976    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:39:40.482190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.482023    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qxn9l\""
Nov 04 02:39:40.482190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.482115    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 04 02:39:40.483575 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.483551    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"]
Nov 04 02:39:40.558805 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.558774    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5lhj\" (UniqueName: \"kubernetes.io/projected/fadfa4ac-e093-4930-9750-0bca1446d2bf-kube-api-access-h5lhj\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.558985 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.558822    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fadfa4ac-e093-4930-9750-0bca1446d2bf-data\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.559296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.559276    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fadfa4ac-e093-4930-9750-0bca1446d2bf-data\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.620758 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.620730    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5lhj\" (UniqueName: \"kubernetes.io/projected/fadfa4ac-e093-4930-9750-0bca1446d2bf-kube-api-access-h5lhj\") pod \"postgres-7cdbf6dc44-7k5jm\" (UID: \"fadfa4ac-e093-4930-9750-0bca1446d2bf\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:40.694360 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:40.694269    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:41.076994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:41.076960    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"]
Nov 04 02:39:41.080690 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:39:41.080657    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfadfa4ac_e093_4930_9750_0bca1446d2bf.slice/crio-f9a0c4163346d539cd91075ca3dc8b7a46b3f0764adfe76fad053ac9c17fc4bf WatchSource:0}: Error finding container f9a0c4163346d539cd91075ca3dc8b7a46b3f0764adfe76fad053ac9c17fc4bf: Status 404 returned error can't find the container with id f9a0c4163346d539cd91075ca3dc8b7a46b3f0764adfe76fad053ac9c17fc4bf
Nov 04 02:39:41.440333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:41.440255    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm" event={"ID":"fadfa4ac-e093-4930-9750-0bca1446d2bf","Type":"ContainerStarted","Data":"f9a0c4163346d539cd91075ca3dc8b7a46b3f0764adfe76fad053ac9c17fc4bf"}
Nov 04 02:39:42.446316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:42.446280    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg" event={"ID":"c325cb7f-d5e1-4efd-84db-0dfec36c5878","Type":"ContainerStarted","Data":"fa932a8dd506caa48f174af54240cfa2c9e8ee52a51481a7a70ece13409e1d62"}
Nov 04 02:39:42.562086 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:42.562035    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-48kqg" podStartSLOduration=1.799284826 podStartE2EDuration="7.56202257s" totalImagesPullingTime="5.762737744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:39:42.561029558 +0000 UTC m=+337.702042965" watchObservedRunningTime="2026-11-04 02:39:42.56202257 +0000 UTC m=+337.703035977"
Nov 04 02:39:48.477240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:48.477207    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm" event={"ID":"fadfa4ac-e093-4930-9750-0bca1446d2bf","Type":"ContainerStarted","Data":"deeea8ae51070dfdd6832085012bb5c354665762ef3734b301c8a6f2c4f50e96"}
Nov 04 02:39:48.477654 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:48.477276    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:48.593060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:48.593000    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm" podStartSLOduration=1.891476113 podStartE2EDuration="8.592987054s" totalImagesPullingTime="6.701510941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:39:48.592597847 +0000 UTC m=+343.733611266" watchObservedRunningTime="2026-11-04 02:39:48.592987054 +0000 UTC m=+343.734000458"
Nov 04 02:39:54.511589 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:54.511555    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-7k5jm"
Nov 04 02:39:58.737664 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.737635    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"]
Nov 04 02:39:58.836693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.836663    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.845283 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.845253    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"]
Nov 04 02:39:58.916487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916457    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916631 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916492    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916631 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916522    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916753 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916694    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916753 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916734    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916848 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916807    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rk5g\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-kube-api-access-2rk5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:58.916900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:58.916859    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018014 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.017983    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018018    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018055    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018118    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018141    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018170    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rk5g\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-kube-api-access-2rk5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018221    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.018923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.018899    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.020598 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.020579    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.020690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.020665    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.020880 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.020860    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.020946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.020894    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.078672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.078644    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.078908 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.078890    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rk5g\" (UniqueName: \"kubernetes.io/projected/b668da5e-cfe9-47fa-9f63-16cdea56c0bc-kube-api-access-2rk5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zch9j\" (UID: \"b668da5e-cfe9-47fa-9f63-16cdea56c0bc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.170829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.170802    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:39:59.604465 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.604437    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"]
Nov 04 02:39:59.607474 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:39:59.607452    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb668da5e_cfe9_47fa_9f63_16cdea56c0bc.slice/crio-039c498ab608db3117655a522f1e40c67e3752b66847e1fa7c321a11069c88ff WatchSource:0}: Error finding container 039c498ab608db3117655a522f1e40c67e3752b66847e1fa7c321a11069c88ff: Status 404 returned error can't find the container with id 039c498ab608db3117655a522f1e40c67e3752b66847e1fa7c321a11069c88ff
Nov 04 02:39:59.609551 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.609521    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:39:59.609632 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:39:59.609587    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:40:00.532419 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.532380    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j" event={"ID":"b668da5e-cfe9-47fa-9f63-16cdea56c0bc","Type":"ContainerStarted","Data":"fa90a25915858a8e9b7d6036fca2ad7387ddf958f546e427a34e188d1e851682"}
Nov 04 02:40:00.532822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.532442    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j" event={"ID":"b668da5e-cfe9-47fa-9f63-16cdea56c0bc","Type":"ContainerStarted","Data":"039c498ab608db3117655a522f1e40c67e3752b66847e1fa7c321a11069c88ff"}
Nov 04 02:40:00.532822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.532564    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:40:00.534303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.534278    6019 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:40:00.534421 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.534322    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j" podUID="b668da5e-cfe9-47fa-9f63-16cdea56c0bc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:40:00.738462 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:00.738405    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j" podStartSLOduration=2.738390633 podStartE2EDuration="2.738390633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:40:00.701467021 +0000 UTC m=+355.842480456" watchObservedRunningTime="2026-11-04 02:40:00.738390633 +0000 UTC m=+355.879404041"
Nov 04 02:40:01.537456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.537432    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j"
Nov 04 02:40:01.760590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.760557    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:40:01.760841 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.760808    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery" containerID="cri-o://0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb" gracePeriod=30
Nov 04 02:40:01.810800 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.810685    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:01.824257 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.824227    6019 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-wm9hl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.24:8080/ready\": dial tcp 10.132.0.24:8080: connect: connection refused" start-of-body=
Nov 04 02:40:01.824394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.824289    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.24:8080/ready\": dial tcp 10.132.0.24:8080: connect: connection refused"
Nov 04 02:40:01.959582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:01.959546    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:02.104224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.104128    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:02.252916 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.252878    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:02.401723 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.401695    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:02.508322 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.508297    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:40:02.542487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.542462    6019 generic.go:356] "Generic (PLEG): container finished" podID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerID="0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb" exitCode=0
Nov 04 02:40:02.542830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.542506    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"
Nov 04 02:40:02.542830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.542543    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" event={"ID":"67122984-cccf-4f01-b0ca-64d7ebd05c70","Type":"ContainerDied","Data":"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"}
Nov 04 02:40:02.542830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.542602    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl" event={"ID":"67122984-cccf-4f01-b0ca-64d7ebd05c70","Type":"ContainerDied","Data":"b4d4e20043730c1dec3c512ae1234947bbb76578b87f95590919122c8e084fe7"}
Nov 04 02:40:02.542830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.542626    6019 scope.go:122] "RemoveContainer" containerID="0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"
Nov 04 02:40:02.545688 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545662    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.545909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545790    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.545909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545827    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.545909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545869    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.546050 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545909    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.546050 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.545954    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.546050 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.546043    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\") pod \"67122984-cccf-4f01-b0ca-64d7ebd05c70\" (UID: \"67122984-cccf-4f01-b0ca-64d7ebd05c70\") "
Nov 04 02:40:02.546205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.546181    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:40:02.547033 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.546478    6019 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-ca-configmap\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.548161 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.548128    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:02.549760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.549733    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:40:02.551452 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.551428    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "kube-api-access-x8l6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:40:02.553463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.553419    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:40:02.553557 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.553452    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:40:02.553946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.553920    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:40:02.554790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.554768    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig" pod "67122984-cccf-4f01-b0ca-64d7ebd05c70" (UID: "67122984-cccf-4f01-b0ca-64d7ebd05c70"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:40:02.568965 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.568944    6019 scope.go:122] "RemoveContainer" containerID="0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"
Nov 04 02:40:02.569285 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:40:02.569265    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb\": container with ID starting with 0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb not found: ID does not exist" containerID="0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"
Nov 04 02:40:02.569348 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.569292    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb"} err="failed to get container status \"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb\": rpc error: code = NotFound desc = could not find container \"0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb\": container with ID starting with 0586b4ce88c31b0eb313749d05519547d47e0d8fa885ddddafd88a2a108147cb not found: ID does not exist"
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.647943    6019 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-kubeconfig\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.647970    6019 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-csr-dns-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.647980    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.647990    6019 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67122984-cccf-4f01-b0ca-64d7ebd05c70-cacerts\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.648007    6019 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67122984-cccf-4f01-b0ca-64d7ebd05c70-local-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.648098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.648020    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8l6q\" (UniqueName: \"kubernetes.io/projected/67122984-cccf-4f01-b0ca-64d7ebd05c70-kube-api-access-x8l6q\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:02.961651 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.961573    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:40:02.966331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.966305    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl"]
Nov 04 02:40:02.980151 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:02.980127    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" path="/var/lib/kubelet/pods/67122984-cccf-4f01-b0ca-64d7ebd05c70/volumes"
Nov 04 02:40:03.697963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:03.697934    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:03.849017 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:03.848986    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:03.992957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:03.992879    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.137788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.137759    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.281311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.281280    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.424293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.424257    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.569371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.569295    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.597477 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.597447    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gpx6l"]
Nov 04 02:40:04.597844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.597829    6019 cpu_manager.go:414] "containerMap: removing container" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery"
Nov 04 02:40:04.597919 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.597846    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery"
Nov 04 02:40:04.597919 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.597915    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="67122984-cccf-4f01-b0ca-64d7ebd05c70" containerName="discovery"
Nov 04 02:40:04.600709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.600689    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.652719 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.652689    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tlcjb\""
Nov 04 02:40:04.652828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.652769    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 02:40:04.652828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.652785    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 04 02:40:04.664396 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.664373    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369c87dc-c619-43ea-a902-b89e7bf348a5-cert\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.664506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.664403    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4pjg\" (UniqueName: \"kubernetes.io/projected/369c87dc-c619-43ea-a902-b89e7bf348a5-kube-api-access-b4pjg\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.702223 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.702203    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 02:40:04.703642 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.703625    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gpx6l"]
Nov 04 02:40:04.717884 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.717852    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:40:04.720733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.720709    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.721371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.721337    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.765279 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.765241    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.765450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.765307    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369c87dc-c619-43ea-a902-b89e7bf348a5-cert\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.765450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.765358    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4pjg\" (UniqueName: \"kubernetes.io/projected/369c87dc-c619-43ea-a902-b89e7bf348a5-kube-api-access-b4pjg\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.765450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.765389    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8slj\" (UniqueName: \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.767939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.767915    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369c87dc-c619-43ea-a902-b89e7bf348a5-cert\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.773230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.773210    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 04 02:40:04.773375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.773358    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vr4lr\""
Nov 04 02:40:04.778567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.778531    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:40:04.829273 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.829205    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4pjg\" (UniqueName: \"kubernetes.io/projected/369c87dc-c619-43ea-a902-b89e7bf348a5-kube-api-access-b4pjg\") pod \"kserve-controller-manager-776cc66d47-gpx6l\" (UID: \"369c87dc-c619-43ea-a902-b89e7bf348a5\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.829550 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.829530    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qvtns"]
Nov 04 02:40:04.831797 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.831782    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:04.866379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.866351    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg9q2\" (UniqueName: \"kubernetes.io/projected/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-kube-api-access-lg9q2\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:04.866509 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.866392    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8slj\" (UniqueName: \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.866509 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.866417    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-tls-certs\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:04.866585 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.866551    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.869133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.869108    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.869229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.869188    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:04.884310 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.884292    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dlrzt\""
Nov 04 02:40:04.884310 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.884294    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 02:40:04.891150 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.891132    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qvtns"]
Nov 04 02:40:04.911937 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.911917    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:04.927702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.927683    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8slj\" (UniqueName: \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\") pod \"llmisvc-controller-manager-869d5dcc5d-bnv5h\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:04.968464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.967796    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg9q2\" (UniqueName: \"kubernetes.io/projected/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-kube-api-access-lg9q2\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:04.968464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:04.967875    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-tls-certs\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:05.019407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.019385    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 02:40:05.029144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.029126    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg9q2\" (UniqueName: \"kubernetes.io/projected/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-kube-api-access-lg9q2\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:05.030679 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.030659    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc44dab-8025-4e79-b9a6-fc9f94f70cd0-tls-certs\") pod \"model-serving-api-75bd89fdfc-qvtns\" (UID: \"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0\") " pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:05.082924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.082852    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vr4lr\""
Nov 04 02:40:05.091589 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.091568    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:05.201667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.201642    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dlrzt\""
Nov 04 02:40:05.210807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.210785    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:05.286445 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.286413    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gpx6l"]
Nov 04 02:40:05.289591 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:40:05.289559    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369c87dc_c619_43ea_a902_b89e7bf348a5.slice/crio-b581a1a1a8df1a17464fde792f563f6be60d4e34161bedb1edfb7c81e62ac839 WatchSource:0}: Error finding container b581a1a1a8df1a17464fde792f563f6be60d4e34161bedb1edfb7c81e62ac839: Status 404 returned error can't find the container with id b581a1a1a8df1a17464fde792f563f6be60d4e34161bedb1edfb7c81e62ac839
Nov 04 02:40:05.290965 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.290949    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:40:05.459478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.459395    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:40:05.463463 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:40:05.463430    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb6dd41ad_4da3_40b5_a4ea_5f5cc777739c.slice/crio-3375e42945a86e06d03090ae3c2d6365c24bcc8754e2d2a98e582f11e2d9866e WatchSource:0}: Error finding container 3375e42945a86e06d03090ae3c2d6365c24bcc8754e2d2a98e582f11e2d9866e: Status 404 returned error can't find the container with id 3375e42945a86e06d03090ae3c2d6365c24bcc8754e2d2a98e582f11e2d9866e
Nov 04 02:40:05.556972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.556933    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" event={"ID":"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c","Type":"ContainerStarted","Data":"3375e42945a86e06d03090ae3c2d6365c24bcc8754e2d2a98e582f11e2d9866e"}
Nov 04 02:40:05.558182 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.558158    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l" event={"ID":"369c87dc-c619-43ea-a902-b89e7bf348a5","Type":"ContainerStarted","Data":"b581a1a1a8df1a17464fde792f563f6be60d4e34161bedb1edfb7c81e62ac839"}
Nov 04 02:40:05.579847 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:05.579821    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qvtns"]
Nov 04 02:40:05.582603 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:40:05.582574    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc44dab_8025_4e79_b9a6_fc9f94f70cd0.slice/crio-a427f010725e0f9501430c5fd035cf48309a82b4abdc5811739a1628ed67b715 WatchSource:0}: Error finding container a427f010725e0f9501430c5fd035cf48309a82b4abdc5811739a1628ed67b715: Status 404 returned error can't find the container with id a427f010725e0f9501430c5fd035cf48309a82b4abdc5811739a1628ed67b715
Nov 04 02:40:06.059972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:06.059938    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:06.565058 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:06.565016    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns" event={"ID":"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0","Type":"ContainerStarted","Data":"a427f010725e0f9501430c5fd035cf48309a82b4abdc5811739a1628ed67b715"}
Nov 04 02:40:06.609211 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:40:06.609172    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:40:06.655163 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:06.655129    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:06.801408 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:06.801370    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:08.576644 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.576612    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns" event={"ID":"fdc44dab-8025-4e79-b9a6-fc9f94f70cd0","Type":"ContainerStarted","Data":"67e58cb586bc15d183ded5e89dcf569c476950f5989fcf69b6f1873bbafa22e2"}
Nov 04 02:40:08.577027 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.576767    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:08.578201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.578175    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" event={"ID":"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c","Type":"ContainerStarted","Data":"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"}
Nov 04 02:40:08.578334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.578241    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:08.579689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.579670    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l" event={"ID":"369c87dc-c619-43ea-a902-b89e7bf348a5","Type":"ContainerStarted","Data":"4e4eaa2747e3272f4cb98b29f288fc78e4b179142a3e6bc63cd0f8318c44f993"}
Nov 04 02:40:08.579802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.579791    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:08.693738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.693651    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns" podStartSLOduration=2.212646349 podStartE2EDuration="4.693638325s" totalImagesPullingTime="2.480991976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:40:08.692626954 +0000 UTC m=+363.833640364" watchObservedRunningTime="2026-11-04 02:40:08.693638325 +0000 UTC m=+363.834651734"
Nov 04 02:40:08.808232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:08.808167    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l" podStartSLOduration=2.007361986 podStartE2EDuration="4.808147656s" totalImagesPullingTime="2.80078567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:40:08.807566587 +0000 UTC m=+363.948579994" watchObservedRunningTime="2026-11-04 02:40:08.808147656 +0000 UTC m=+363.949161066"
Nov 04 02:40:09.359380 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:09.359348    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:09.503428 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:09.503378    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:10.420718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:10.420647    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:10.686425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:10.686356    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:12.496527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:12.496489    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:12.638980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:12.638948    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:12.780300 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:12.780267    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:12.926008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:12.925975    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.071696 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.071624    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.218482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.218446    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.366651 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.366575    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.509295 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.509264    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.651266 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.651139    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:13.795641 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:13.795588    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:19.589808 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:19.589781    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-qvtns"
Nov 04 02:40:19.709888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:19.709819    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" podStartSLOduration=12.760562137 podStartE2EDuration="15.709801513s" totalImagesPullingTime="2.949239376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:40:08.925282474 +0000 UTC m=+364.066295881" watchObservedRunningTime="2026-11-04 02:40:19.709801513 +0000 UTC m=+374.850814920"
Nov 04 02:40:26.375686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:26.375655    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:36.602230 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:40:36.602192    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:40:39.587334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:39.587244    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:40:39.590523 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:39.590503    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-gpx6l"
Nov 04 02:40:40.073459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:40.073427    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:40.217474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:40.217443    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:42.218333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:42.218296    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:42.384696 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:42.384666    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:42.534301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:42.534272    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:42.677276 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:42.677239    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:55.090284 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.090249    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:55.384971 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.384886    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:40:55.653010 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.652912    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 02:40:55.655378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.655357    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.709090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.709050    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.709239 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.709104    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.709239 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.709211    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99trv\" (UniqueName: \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.709368 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.709268    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.758561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.758528    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 02:40:55.758701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.758591    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 02:40:55.758701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.758692    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qtjr2\""
Nov 04 02:40:55.759804 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.759783    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 02:40:55.810305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.810274    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.810466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.810370    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.810466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.810395    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.810466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.810432    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99trv\" (UniqueName: \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.811269 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.811242    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.813099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.813050    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.813260 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.813241    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.874276 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.874244    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99trv\" (UniqueName: \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\") pod \"maas-api-79b4f86cc7-5rqnn\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") " pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:55.966035 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:55.965973    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:56.341478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:56.341445    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 02:40:56.344789 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:40:56.344748    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5971c81b_e937_4ae2_b565_9348e7762e36.slice/crio-8c9c3437517e34f25eb2b00827591f906c186a36470c6e2dd9dfb9e202ce82a1 WatchSource:0}: Error finding container 8c9c3437517e34f25eb2b00827591f906c186a36470c6e2dd9dfb9e202ce82a1: Status 404 returned error can't find the container with id 8c9c3437517e34f25eb2b00827591f906c186a36470c6e2dd9dfb9e202ce82a1
Nov 04 02:40:56.795211 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:56.795176    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" event={"ID":"5971c81b-e937-4ae2-b565-9348e7762e36","Type":"ContainerStarted","Data":"8c9c3437517e34f25eb2b00827591f906c186a36470c6e2dd9dfb9e202ce82a1"}
Nov 04 02:40:59.245043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.245019    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:40:59.245850 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.245820    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" podUID="474f04ff-08a1-476b-8a8b-a81a6160d185" containerName="authorino" containerID="cri-o://cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9" gracePeriod=30
Nov 04 02:40:59.502036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.502011    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:40:59.538953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.538926    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\" (UniqueName: \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\") pod \"474f04ff-08a1-476b-8a8b-a81a6160d185\" (UID: \"474f04ff-08a1-476b-8a8b-a81a6160d185\") "
Nov 04 02:40:59.540957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.540935    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5" pod "474f04ff-08a1-476b-8a8b-a81a6160d185" (UID: "474f04ff-08a1-476b-8a8b-a81a6160d185"). InnerVolumeSpecName "kube-api-access-dpqd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:40:59.639992 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.639961    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpqd5\" (UniqueName: \"kubernetes.io/projected/474f04ff-08a1-476b-8a8b-a81a6160d185-kube-api-access-dpqd5\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:40:59.810311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.810274    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" event={"ID":"5971c81b-e937-4ae2-b565-9348e7762e36","Type":"ContainerStarted","Data":"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"}
Nov 04 02:40:59.810507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.810382    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:40:59.811407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.811383    6019 generic.go:356] "Generic (PLEG): container finished" podID="474f04ff-08a1-476b-8a8b-a81a6160d185" containerID="cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9" exitCode=0
Nov 04 02:40:59.811506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.811418    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" event={"ID":"474f04ff-08a1-476b-8a8b-a81a6160d185","Type":"ContainerDied","Data":"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"}
Nov 04 02:40:59.811506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.811443    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf" event={"ID":"474f04ff-08a1-476b-8a8b-a81a6160d185","Type":"ContainerDied","Data":"65e230cc689e4e094e784c856d4ed99996203cbb3144ef2310318ace774cf268"}
Nov 04 02:40:59.811506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.811419    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-rxglf"
Nov 04 02:40:59.811506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.811460    6019 scope.go:122] "RemoveContainer" containerID="cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"
Nov 04 02:40:59.820102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.820083    6019 scope.go:122] "RemoveContainer" containerID="cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"
Nov 04 02:40:59.820387 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:40:59.820370    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9\": container with ID starting with cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9 not found: ID does not exist" containerID="cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"
Nov 04 02:40:59.820453 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.820393    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9"} err="failed to get container status \"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9\": rpc error: code = NotFound desc = could not find container \"cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9\": container with ID starting with cf81c765af82318410ee31df4af55888043748947799084578f0f74fe465e5d9 not found: ID does not exist"
Nov 04 02:40:59.930586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:40:59.930539    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" podStartSLOduration=2.092148999 podStartE2EDuration="4.930526757s" totalImagesPullingTime="2.838377758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:40:59.930263418 +0000 UTC m=+415.071276828" watchObservedRunningTime="2026-11-04 02:40:59.930526757 +0000 UTC m=+415.071540164"
Nov 04 02:41:00.042555 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:00.042517    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:41:00.051101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:00.051056    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-rxglf"]
Nov 04 02:41:00.980142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:00.980108    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="474f04ff-08a1-476b-8a8b-a81a6160d185" path="/var/lib/kubelet/pods/474f04ff-08a1-476b-8a8b-a81a6160d185/volumes"
Nov 04 02:41:05.824065 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:05.824038    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 02:41:06.602274 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:41:06.602233    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:41:16.291362 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:16.291327    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:41:17.109830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:17.109799    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:41:20.339324 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.339285    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"]
Nov 04 02:41:20.339702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.339656    6019 cpu_manager.go:414] "containerMap: removing container" podUID="474f04ff-08a1-476b-8a8b-a81a6160d185" containerName="authorino"
Nov 04 02:41:20.339702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.339667    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="474f04ff-08a1-476b-8a8b-a81a6160d185" containerName="authorino"
Nov 04 02:41:20.339769 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.339742    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="474f04ff-08a1-476b-8a8b-a81a6160d185" containerName="authorino"
Nov 04 02:41:20.342857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.342838    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.400732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.400700    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"]
Nov 04 02:41:20.423666 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.423631    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkwgv\" (UniqueName: \"kubernetes.io/projected/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-kube-api-access-zkwgv\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.423797 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.423681    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-cert\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.524207 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.524175    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwgv\" (UniqueName: \"kubernetes.io/projected/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-kube-api-access-zkwgv\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.524366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.524217    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-cert\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.526818 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.526797    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-cert\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.585519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.585489    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkwgv\" (UniqueName: \"kubernetes.io/projected/a72b7dac-d9ce-4a3b-9557-9db5bab724f6-kube-api-access-zkwgv\") pod \"llmisvc-controller-manager-6466c554d8-gbv2g\" (UID: \"a72b7dac-d9ce-4a3b-9557-9db5bab724f6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:20.655377 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:20.655301    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:21.024564 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:21.024533    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"]
Nov 04 02:41:21.031879 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:41:21.031712    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72b7dac_d9ce_4a3b_9557_9db5bab724f6.slice/crio-6df714dca6ea9cb97468b745243ab16e1bf9692b2e4f410e5185d98bf97f5671 WatchSource:0}: Error finding container 6df714dca6ea9cb97468b745243ab16e1bf9692b2e4f410e5185d98bf97f5671: Status 404 returned error can't find the container with id 6df714dca6ea9cb97468b745243ab16e1bf9692b2e4f410e5185d98bf97f5671
Nov 04 02:41:21.910794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:21.910759    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g" event={"ID":"a72b7dac-d9ce-4a3b-9557-9db5bab724f6","Type":"ContainerStarted","Data":"b9fa169d49638afb7c39d77cca7657e5568c76bc2f40c78a15c24c818d6869ba"}
Nov 04 02:41:21.911247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:21.910847    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g" event={"ID":"a72b7dac-d9ce-4a3b-9557-9db5bab724f6","Type":"ContainerStarted","Data":"6df714dca6ea9cb97468b745243ab16e1bf9692b2e4f410e5185d98bf97f5671"}
Nov 04 02:41:21.911247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:21.910968    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:22.028383 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:22.028283    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g" podStartSLOduration=1.307228479 podStartE2EDuration="2.028270012s" totalImagesPullingTime="721.041533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:41:22.026778109 +0000 UTC m=+437.167791517" watchObservedRunningTime="2026-11-04 02:41:22.028270012 +0000 UTC m=+437.169283418"
Nov 04 02:41:26.724223 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:26.724188    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:41:36.602274 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:41:36.602233    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:41:52.917675 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:52.917648    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gbv2g"
Nov 04 02:41:53.085334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.085304    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:41:53.085595 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.085554    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" podUID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" containerName="manager" containerID="cri-o://77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb" gracePeriod=30
Nov 04 02:41:53.326857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.326833    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:41:53.412935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.412895    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\" (UniqueName: \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\") pod \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") "
Nov 04 02:41:53.413063 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.412975    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\" (UniqueName: \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\") pod \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\" (UID: \"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c\") "
Nov 04 02:41:53.415224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.415196    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj" pod "b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" (UID: "b6dd41ad-4da3-40b5-a4ea-5f5cc777739c"). InnerVolumeSpecName "kube-api-access-q8slj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:41:53.415319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.415253    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert" pod "b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" (UID: "b6dd41ad-4da3-40b5-a4ea-5f5cc777739c"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:41:53.514384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.514349    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8slj\" (UniqueName: \"kubernetes.io/projected/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-kube-api-access-q8slj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:41:53.514384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.514384    6019 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:41:53.914944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:53.914867    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:41:54.050912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.050878    6019 generic.go:356] "Generic (PLEG): container finished" podID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" containerID="77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb" exitCode=0
Nov 04 02:41:54.051371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.050934    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"
Nov 04 02:41:54.051371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.050967    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" event={"ID":"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c","Type":"ContainerDied","Data":"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"}
Nov 04 02:41:54.051371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.051022    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h" event={"ID":"b6dd41ad-4da3-40b5-a4ea-5f5cc777739c","Type":"ContainerDied","Data":"3375e42945a86e06d03090ae3c2d6365c24bcc8754e2d2a98e582f11e2d9866e"}
Nov 04 02:41:54.051371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.051046    6019 scope.go:122] "RemoveContainer" containerID="77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"
Nov 04 02:41:54.058676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.058648    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:41:54.060438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.060425    6019 scope.go:122] "RemoveContainer" containerID="77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"
Nov 04 02:41:54.060705 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:41:54.060687    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb\": container with ID starting with 77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb not found: ID does not exist" containerID="77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"
Nov 04 02:41:54.060787 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.060713    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb"} err="failed to get container status \"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb\": rpc error: code = NotFound desc = could not find container \"77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb\": container with ID starting with 77d71e0aa9a140e2ddc6336446b23d66f24c30df3c24a1269f0970bcd1d871fb not found: ID does not exist"
Nov 04 02:41:54.213156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.213081    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:41:54.262724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.262694    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bnv5h"]
Nov 04 02:41:54.980278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:41:54.980247    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" path="/var/lib/kubelet/pods/b6dd41ad-4da3-40b5-a4ea-5f5cc777739c/volumes"
Nov 04 02:42:06.604355 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:42:06.604315    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:42:11.168516 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:11.168475    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:17.587471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:17.587433    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:21.493466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:21.493434    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:25.377042 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:25.377004    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:28.082503 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:28.082468    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:30.182264 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:30.182227    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:35.423282 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:35.423247    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:36.602498 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:42:36.602412    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:42:39.806519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:39.806488    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:43.186951 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:43.186914    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:45.029627 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.029596    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:42:45.030016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.029954    6019 cpu_manager.go:414] "containerMap: removing container" podUID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" containerName="manager"
Nov 04 02:42:45.030016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.029966    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" containerName="manager"
Nov 04 02:42:45.030128 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.030046    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6dd41ad-4da3-40b5-a4ea-5f5cc777739c" containerName="manager"
Nov 04 02:42:45.033038 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.033018    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.085481 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.085444    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pm6wh\""
Nov 04 02:42:45.136461 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.136436    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:42:45.164351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164324    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164365    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164456    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164485    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164513    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164591    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164633    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164695    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6cpj\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.164749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.164718    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.265792 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265759    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.265792 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265797    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265832    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265863    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265894    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265914    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265936    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.265979    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6cpj\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.266010    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.266247    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.266480    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.266495    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.266618 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.266531    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.267133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.267107    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.268359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.268339    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.268692 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.268675    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.327379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.327311    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.327379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.327329    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6cpj\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\") pod \"maas-default-gateway-openshift-default-76597d6855-ts4v7\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.345164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.345141    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:45.492231 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.492202    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:45.639315 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.639240    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:45.769940 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:45.769912    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:42:45.772178 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:42:45.772150    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaab526c_8435_496c_973f_3bf4318bb392.slice/crio-52730cf25c5aaea5977cb72dfd33fd71a52e034e8decc2b1c725d31187061a1a WatchSource:0}: Error finding container 52730cf25c5aaea5977cb72dfd33fd71a52e034e8decc2b1c725d31187061a1a: Status 404 returned error can't find the container with id 52730cf25c5aaea5977cb72dfd33fd71a52e034e8decc2b1c725d31187061a1a
Nov 04 02:42:46.279756 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:46.279724    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" event={"ID":"caab526c-8435-496c-973f-3bf4318bb392","Type":"ContainerStarted","Data":"52730cf25c5aaea5977cb72dfd33fd71a52e034e8decc2b1c725d31187061a1a"}
Nov 04 02:42:46.627398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:46.627316    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:49.968562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:49.968530    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:42:49.968910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:49.968591    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:42:49.968910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:49.968622    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:42:50.309054 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:50.309014    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" event={"ID":"caab526c-8435-496c-973f-3bf4318bb392","Type":"ContainerStarted","Data":"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"}
Nov 04 02:42:50.346241 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:50.346195    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:42:50.512604 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:50.512543    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podStartSLOduration=2.31900027 podStartE2EDuration="6.512523101s" totalImagesPullingTime="4.193522831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:42:50.479259429 +0000 UTC m=+525.620272837" watchObservedRunningTime="2026-11-04 02:42:50.512523101 +0000 UTC m=+525.653536509"
Nov 04 02:42:50.526605 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:50.526581    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:51.347092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:51.347039    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:51.347458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:51.347109    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:52.345921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:52.345888    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:52.346125 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:52.345941    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:53.346203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:53.346171    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:53.346599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:53.346222    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:54.346471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:54.346438    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:54.346851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:54.346492    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:55.345548 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:55.345509    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:55.345719 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:55.345562    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:56.346156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:56.346121    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Nov 04 02:42:56.346660 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:56.346194    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Nov 04 02:42:58.346782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:58.346742    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:42:58.347301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:58.346805    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:42:58.396993 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:58.396959    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:58.544186 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:58.544152    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:42:59.388265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:59.388222    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:42:59.388766 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:42:59.388307    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:43:00.110706 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:00.110667    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:43:00.111062 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:00.111025    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:43:00.111971 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:00.111948    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:43:06.604929 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:43:06.604888    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:43:18.533190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:18.533151    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:18.688060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:18.688025    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:27.844782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:27.844744    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:28.019611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:28.019581    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:28.436333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:28.436297    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:30.545427 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:30.545394    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:36.602390 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:43:36.602352    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:43:40.376794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:40.376712    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:40.521957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:40.521897    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:51.528340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:51.528305    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:52.084190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:52.084158    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:53.661293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:53.661250    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:53.808994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:53.808952    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:55.408244 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:55.408208    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:43:55.563286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:43:55.563254    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:00.607012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.606976    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:44:00.610627 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.610607    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663421    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663605 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663463    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663605 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663556    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88d4h\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663605 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663595    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663743 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663619    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663743 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663679    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663849 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663802    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663871    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.663939 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.663914    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.712055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.712021    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:44:00.764386 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764356    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764386 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764397    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764634 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764418    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764634 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764435    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764634 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764573    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764634 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764626    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764832 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764685    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764832 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764715    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.764832 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764786    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88d4h\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.765035 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764961    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.765035 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.764992    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.765187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.765115    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.765187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.765159    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.765504 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.765483    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.767048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.767020    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.767195 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.767179    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.830465 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.830436    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.830623 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.830605    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88d4h\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\") pod \"maas-default-gateway-openshift-default-76597d6855-z29xk\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:00.923520 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:00.923442    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:01.348025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.347990    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:44:01.350315 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:44:01.350290    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod106f0b36_091f_49d8_af2d_e8e46d131645.slice/crio-798f679718e2a9e455a8562d31af97305268f848c171e8d983ced9e71fd5b7c3 WatchSource:0}: Error finding container 798f679718e2a9e455a8562d31af97305268f848c171e8d983ced9e71fd5b7c3: Status 404 returned error can't find the container with id 798f679718e2a9e455a8562d31af97305268f848c171e8d983ced9e71fd5b7c3
Nov 04 02:44:01.352512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.352484    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:44:01.352598 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.352548    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:44:01.352598 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.352577    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:44:01.639496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.639409    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" event={"ID":"106f0b36-091f-49d8-af2d-e8e46d131645","Type":"ContainerStarted","Data":"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"}
Nov 04 02:44:01.640109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.640061    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" event={"ID":"106f0b36-091f-49d8-af2d-e8e46d131645","Type":"ContainerStarted","Data":"798f679718e2a9e455a8562d31af97305268f848c171e8d983ced9e71fd5b7c3"}
Nov 04 02:44:01.763109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.763036    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podStartSLOduration=1.763020113 podStartE2EDuration="1.763020113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:44:01.761134329 +0000 UTC m=+596.902147747" watchObservedRunningTime="2026-11-04 02:44:01.763020113 +0000 UTC m=+596.904033525"
Nov 04 02:44:01.924264 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:01.924179    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:02.925436 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:02.925401    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:02.925843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:02.925453    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:03.923936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:03.923897    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:03.924152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:03.923956    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:04.924392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:04.924353    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:04.924809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:04.924420    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:05.037521 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:05.037498    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:44:05.039692 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:05.039670    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:44:05.924201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:05.924162    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:05.924374 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:05.924231    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:06.602976 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:44:06.602921    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:44:06.924014 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:06.923928    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:06.924014 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:06.924001    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:07.924065 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:07.924025    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:07.924713 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:07.924098    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:08.924245 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:08.924213    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-z29xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 04 02:44:08.924740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:08.924301    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 04 02:44:09.955615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:09.955577    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:09.956145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:09.956011    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:09.956749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:09.956734    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:44:28.331630 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:28.331595    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:28.479354 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:28.479306    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:28.810011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:28.809977    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.065308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.065221    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.298254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.298220    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.449930 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.449818    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.599228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.599199    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.745667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.745567    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:29.889903 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:29.889859    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.036103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.036045    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.203925 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.203887    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.377339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.377257    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.548442 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.548399    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.699118 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.698991    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:30.908590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:30.908555    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:31.059444 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:31.059409    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:31.209469 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:31.209436    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:33.169147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:33.169108    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:33.348190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:33.348160    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:33.493191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:33.493114    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:33.991480 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:33.991452    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:34.195562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:34.195507    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:34.346446 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:34.346410    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:35.137120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:35.137089    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:35.358947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:35.358914    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:35.534247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:35.534219    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:36.602301 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:44:36.602267    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:44:36.871372 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:36.871286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:37.142934 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:37.142858    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:37.314861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:37.314829    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:37.457121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:37.457023    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:37.633575 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:37.633539    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:38.743961 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:38.743932    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:38.929727 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:38.929700    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:39.174318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:39.174288    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:39.386720 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:39.386689    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:39.604185 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:39.604148    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:39.766143 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:39.766112    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:39.934750 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:39.934654    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:40.082627 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:40.082584    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:40.231225 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:40.231145    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:42.478157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:42.478113    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:42.650530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:42.650501    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:44.957673 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:44.957640    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 02:44:44.961404 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:44.961380    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.013524 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.013495    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-fe4e0c-serving-cert\""
Nov 04 02:44:45.064301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.064273    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 02:44:45.077918 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.077893    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.078038 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.077933    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.078038 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.077957    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fqlk\" (UniqueName: \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.078152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.078037    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.156390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.156348    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:45.179171 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.179145    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.179285 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.179177    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.179285 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.179201    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fqlk\" (UniqueName: \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.179285 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.179221    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.179446 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:44:45.179311    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-serving-cert: secret "maas-api-e2e-worker-w0-fe4e0c-serving-cert" not found
Nov 04 02:44:45.179446 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:44:45.179387    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls podName:efe32154-a1cf-4ba0-a775-a95448276eae nodeName:}" failed. No retries permitted until 2026-11-04 02:44:45.679365795 +0000 UTC m=+640.820379197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls") pod "maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" (UID: "efe32154-a1cf-4ba0-a775-a95448276eae") : secret "maas-api-e2e-worker-w0-fe4e0c-serving-cert" not found
Nov 04 02:44:45.179894 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.179834    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.181810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.181789    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.240690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.240616    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fqlk\" (UniqueName: \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.324765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.324729    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:45.465920 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.465884    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:45.686034 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.686002    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.688547 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.688519    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:45.873663 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:45.873630    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:46.243947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:46.243921    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 02:44:46.247409 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:44:46.247383    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe32154_a1cf_4ba0_a775_a95448276eae.slice/crio-7b3bcadc5b9db43ac81aa15be1d5a84ceebb392d2546d5c7b15a2c4c630bf474 WatchSource:0}: Error finding container 7b3bcadc5b9db43ac81aa15be1d5a84ceebb392d2546d5c7b15a2c4c630bf474: Status 404 returned error can't find the container with id 7b3bcadc5b9db43ac81aa15be1d5a84ceebb392d2546d5c7b15a2c4c630bf474
Nov 04 02:44:46.881520 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:46.881482    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" event={"ID":"efe32154-a1cf-4ba0-a775-a95448276eae","Type":"ContainerStarted","Data":"7b3bcadc5b9db43ac81aa15be1d5a84ceebb392d2546d5c7b15a2c4c630bf474"}
Nov 04 02:44:47.361421 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:47.361376    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:47.544426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:47.544349    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:47.887515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:47.887432    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" event={"ID":"efe32154-a1cf-4ba0-a775-a95448276eae","Type":"ContainerStarted","Data":"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"}
Nov 04 02:44:47.887686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:47.887511    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:48.005948 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:48.005756    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" podStartSLOduration=3.013983499 podStartE2EDuration="4.005667316s" totalImagesPullingTime="991.683817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:44:48.004029816 +0000 UTC m=+643.145043223" watchObservedRunningTime="2026-11-04 02:44:48.005667316 +0000 UTC m=+643.146680723"
Nov 04 02:44:49.874509 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:49.874474    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:50.067203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.067134    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:50.442424 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.442393    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 02:44:50.446168 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.446146    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.547572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.547543    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-3b7476-dockercfg-4h9db\""
Nov 04 02:44:50.547572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.547562    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-3b7476\""
Nov 04 02:44:50.548907 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.548886    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 02:44:50.630799 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.630771    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:50.634892 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.634861    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8hm\" (UniqueName: \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.635023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.634902    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.736235 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.736146    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8hm\" (UniqueName: \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.736235 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.736192    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.736849 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.736829    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:50.797340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:50.797309    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8hm\" (UniqueName: \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\") pod \"payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:51.059064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:51.059032    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:51.431413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:51.431391    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 02:44:51.433945 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:44:51.433917    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d45294_a924_42cf_8e63_c31279716771.slice/crio-a686a677d366b715dacb02fcfde3047008fc7ba12c34271bf054d53c9c6cd1ee WatchSource:0}: Error finding container a686a677d366b715dacb02fcfde3047008fc7ba12c34271bf054d53c9c6cd1ee: Status 404 returned error can't find the container with id a686a677d366b715dacb02fcfde3047008fc7ba12c34271bf054d53c9c6cd1ee
Nov 04 02:44:51.906271 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:51.906235    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" event={"ID":"76d45294-a924-42cf-8e63-c31279716771","Type":"ContainerStarted","Data":"a686a677d366b715dacb02fcfde3047008fc7ba12c34271bf054d53c9c6cd1ee"}
Nov 04 02:44:52.419213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:52.419186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:52.597183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:52.597149    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:53.898280 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:53.898238    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 02:44:54.931615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:54.931583    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:55.115333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:55.115299    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:55.923986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:55.923935    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:55.929345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:55.929315    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" event={"ID":"76d45294-a924-42cf-8e63-c31279716771","Type":"ContainerStarted","Data":"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"}
Nov 04 02:44:55.929507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:55.929420    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:44:56.049803 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:56.049748    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" podStartSLOduration=2.257298231 podStartE2EDuration="6.049735586s" totalImagesPullingTime="3.792437355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:44:56.048909506 +0000 UTC m=+651.189922912" watchObservedRunningTime="2026-11-04 02:44:56.049735586 +0000 UTC m=+651.190748993"
Nov 04 02:44:56.089923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:56.089882    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:56.234823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:56.234734    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:44:59.884822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:44:59.884786    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:00.071119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.071059    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:00.178694 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.178610    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:00.183265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.183239    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.235715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.235679    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slbq8\""
Nov 04 02:45:00.239764 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.239733    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.239911 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.239795    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wl99\" (UniqueName: \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.286746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.286715    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:00.302157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.302124    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 02:45:00.305542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.305519    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.340363 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.340329    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wl99\" (UniqueName: \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.340489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.340375    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.340489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.340413    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l82l\" (UniqueName: \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.340489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.340445    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.341028 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.341010    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.362243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.362219    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-ebb7fd-dockercfg-6pcjs\""
Nov 04 02:45:00.365402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.365386    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-ebb7fd\""
Nov 04 02:45:00.382226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.382202    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 02:45:00.401849 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.401816    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wl99\" (UniqueName: \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\") pod \"maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.441579 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.441492    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.441579 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.441570    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l82l\" (UniqueName: \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.442518 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.442485    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.499458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.499422    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:00.503198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.503172    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l82l\" (UniqueName: \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\") pod \"payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.618529 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.618489    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:00.874213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.874183    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:00.878287 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:45:00.878250    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24b5eeb_9d22_41b2_8962_8c4397825662.slice/crio-15bd2bbcd1a73003374fcd06f565514cbf5677e59549edef0cab4097e87d090d WatchSource:0}: Error finding container 15bd2bbcd1a73003374fcd06f565514cbf5677e59549edef0cab4097e87d090d: Status 404 returned error can't find the container with id 15bd2bbcd1a73003374fcd06f565514cbf5677e59549edef0cab4097e87d090d
Nov 04 02:45:00.962415 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.962384    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerStarted","Data":"15bd2bbcd1a73003374fcd06f565514cbf5677e59549edef0cab4097e87d090d"}
Nov 04 02:45:00.989957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:00.989926    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 02:45:00.993140 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:45:00.993111    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588ab8e8_fc03_4c32_bd6c_af484aa51afc.slice/crio-e053daffb0a8991c02f72699222bf6c21ecd4abcb787c452a93e6dc2679771db WatchSource:0}: Error finding container e053daffb0a8991c02f72699222bf6c21ecd4abcb787c452a93e6dc2679771db: Status 404 returned error can't find the container with id e053daffb0a8991c02f72699222bf6c21ecd4abcb787c452a93e6dc2679771db
Nov 04 02:45:01.254272 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.254233    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 02:45:01.257914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.257890    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.310183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.310147    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-3b7476-serving-cert\""
Nov 04 02:45:01.317549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.317516    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 02:45:01.350948 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.350912    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.351131 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.350954    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.351171 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.351129    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.351171 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.351150    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tps25\" (UniqueName: \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.451946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.451906    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.451946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.451947    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.452189 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:01.452055    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-serving-cert: secret "maas-api-e2e-worker-w5-3b7476-serving-cert" not found
Nov 04 02:45:01.452189 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.452124    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.452189 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:01.452143    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls podName:70029146-61e2-4890-a9f9-583ef339a4c9 nodeName:}" failed. No retries permitted until 2026-11-04 02:45:01.952125889 +0000 UTC m=+657.093139293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls") pod "maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" (UID: "70029146-61e2-4890-a9f9-583ef339a4c9") : secret "maas-api-e2e-worker-w5-3b7476-serving-cert" not found
Nov 04 02:45:01.452189 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.452172    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tps25\" (UniqueName: \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.452724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.452706    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.454754 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.454737    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.461250 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.461220    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:01.516714 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.516633    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tps25\" (UniqueName: \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.628711 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.628681    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:01.776314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.776234    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:01.956976 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.956948    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.959766 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.959741    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:01.978434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.978402    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" event={"ID":"588ab8e8-fc03-4c32-bd6c-af484aa51afc","Type":"ContainerStarted","Data":"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"}
Nov 04 02:45:01.978900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.978452    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" event={"ID":"588ab8e8-fc03-4c32-bd6c-af484aa51afc","Type":"ContainerStarted","Data":"e053daffb0a8991c02f72699222bf6c21ecd4abcb787c452a93e6dc2679771db"}
Nov 04 02:45:01.978900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:01.978504    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:02.094914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:02.094819    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" podStartSLOduration=2.094804478 podStartE2EDuration="2.094804478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:45:02.093999987 +0000 UTC m=+657.235013394" watchObservedRunningTime="2026-11-04 02:45:02.094804478 +0000 UTC m=+657.235817884"
Nov 04 02:45:02.171010 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:02.170981    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:02.557432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:02.557406    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 02:45:02.560041 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:45:02.560013    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70029146_61e2_4890_a9f9_583ef339a4c9.slice/crio-67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174 WatchSource:0}: Error finding container 67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174: Status 404 returned error can't find the container with id 67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174
Nov 04 02:45:02.983484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:02.983397    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" event={"ID":"70029146-61e2-4890-a9f9-583ef339a4c9","Type":"ContainerStarted","Data":"67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174"}
Nov 04 02:45:03.990681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:03.990590    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" event={"ID":"70029146-61e2-4890-a9f9-583ef339a4c9","Type":"ContainerStarted","Data":"f71b316d956971e61a2e279fc11feff4fb46fbbea75816c2ede5e137a6c57417"}
Nov 04 02:45:03.990681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:03.990653    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:06.008615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:06.008580    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerStarted","Data":"bd880c79575195c19a7aa6210c7ba84b4b74ee37969815652bf2c661ed5a2721"}
Nov 04 02:45:06.124585 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:06.124489    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" podStartSLOduration=4.113130234 podStartE2EDuration="5.124475001s" totalImagesPullingTime="1.011344767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:45:04.110259984 +0000 UTC m=+659.251273748" watchObservedRunningTime="2026-11-04 02:45:06.124475001 +0000 UTC m=+661.265488407"
Nov 04 02:45:06.125480 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:06.125450    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" podStartSLOduration=1.413421088 podStartE2EDuration="6.12543887s" totalImagesPullingTime="4.712017782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:45:06.123513484 +0000 UTC m=+661.264526892" watchObservedRunningTime="2026-11-04 02:45:06.12543887 +0000 UTC m=+661.266452277"
Nov 04 02:45:06.601978 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:06.601939    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:45:06.938737 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:06.938661    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 02:45:06.956032 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:06.956000    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:07.024972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:07.024942    6019 generic.go:356] "Generic (PLEG): container finished" podID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerID="bd880c79575195c19a7aa6210c7ba84b4b74ee37969815652bf2c661ed5a2721" exitCode=60
Nov 04 02:45:07.025489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:07.025016    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerDied","Data":"bd880c79575195c19a7aa6210c7ba84b4b74ee37969815652bf2c661ed5a2721"}
Nov 04 02:45:07.025489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:07.025367    6019 scope.go:122] "RemoveContainer" containerID="bd880c79575195c19a7aa6210c7ba84b4b74ee37969815652bf2c661ed5a2721"
Nov 04 02:45:08.031310 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:08.031273    6019 generic.go:356] "Generic (PLEG): container finished" podID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerID="c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b" exitCode=60
Nov 04 02:45:08.031712 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:08.031357    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerDied","Data":"c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b"}
Nov 04 02:45:08.031712 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:08.031423    6019 scope.go:122] "RemoveContainer" containerID="bd880c79575195c19a7aa6210c7ba84b4b74ee37969815652bf2c661ed5a2721"
Nov 04 02:45:08.031712 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:08.031686    6019 scope.go:122] "RemoveContainer" containerID="c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b"
Nov 04 02:45:08.031944 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:08.031925    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj_odh-ai-gateway-infra(e24b5eeb-9d22-41b2-8962-8c4397825662)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662"
Nov 04 02:45:09.038150 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:09.038122    6019 scope.go:122] "RemoveContainer" containerID="c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b"
Nov 04 02:45:09.038580 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:09.038374    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj_odh-ai-gateway-infra(e24b5eeb-9d22-41b2-8962-8c4397825662)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662"
Nov 04 02:45:10.002242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:10.002216    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 02:45:11.898305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.898269    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 02:45:11.903385 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.903367    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:11.955898 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.955872    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-e516ff-serving-cert\""
Nov 04 02:45:11.960430 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.960400    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:11.960606 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.960571    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:11.960706 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.960689    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7sx8\" (UniqueName: \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:11.960760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:11.960725    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.007020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.006984    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 02:45:12.062137 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.062106    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.062308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.062150    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.062308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.062222    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.062308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.062278    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7sx8\" (UniqueName: \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.062479 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:12.062313    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-serving-cert: secret "maas-api-e2e-worker-w1-e516ff-serving-cert" not found
Nov 04 02:45:12.062479 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:12.062388    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls podName:6a59a675-9277-4c29-8c2a-7be0f35f754c nodeName:}" failed. No retries permitted until 2026-11-04 02:45:12.56236465 +0000 UTC m=+667.703378053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls") pod "maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" (UID: "6a59a675-9277-4c29-8c2a-7be0f35f754c") : secret "maas-api-e2e-worker-w1-e516ff-serving-cert" not found
Nov 04 02:45:12.062990 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.062959    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.064709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.064692    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.096112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.096047    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:12.123703 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.123673    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7sx8\" (UniqueName: \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.264679 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.264552    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:12.467337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.467300    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:12.567570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.567482    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.570120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.570100    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.815954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.815921    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:12.986909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:12.986879    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 02:45:13.268737 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:13.268604    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 02:45:13.271043 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:45:13.271017    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a59a675_9277_4c29_8c2a_7be0f35f754c.slice/crio-e6b3ccb6b206166a7aa7c0a7b9447a16b9a31d7397de4ecf548f297d12eaaef4 WatchSource:0}: Error finding container e6b3ccb6b206166a7aa7c0a7b9447a16b9a31d7397de4ecf548f297d12eaaef4: Status 404 returned error can't find the container with id e6b3ccb6b206166a7aa7c0a7b9447a16b9a31d7397de4ecf548f297d12eaaef4
Nov 04 02:45:13.272316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:13.272300    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:45:13.828548 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:13.828517    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:14.061721 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:14.061687    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" event={"ID":"6a59a675-9277-4c29-8c2a-7be0f35f754c","Type":"ContainerStarted","Data":"e6b3ccb6b206166a7aa7c0a7b9447a16b9a31d7397de4ecf548f297d12eaaef4"}
Nov 04 02:45:14.463163 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:14.463135    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:14.618909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:14.618880    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:14.767335 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:14.767299    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:15.068458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:15.068384    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" event={"ID":"6a59a675-9277-4c29-8c2a-7be0f35f754c","Type":"ContainerStarted","Data":"13547f0ac84862898ac0bbf78a777efc500cded4331931ba12e8ca8c17480111"}
Nov 04 02:45:15.068458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:15.068456    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:15.185948 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:15.185896    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" podStartSLOduration=3.05374197 podStartE2EDuration="4.185880833s" totalImagesPullingTime="1.132138863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:45:15.185561119 +0000 UTC m=+670.326574542" watchObservedRunningTime="2026-11-04 02:45:15.185880833 +0000 UTC m=+670.326894237"
Nov 04 02:45:17.546912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:17.546866    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:19.159977 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.159942    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 02:45:19.163784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.163767    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.219614 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.219589    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-cdcf67\"/\"kube-root-ca.crt\""
Nov 04 02:45:19.249974 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.249940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:19.269174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.269143    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-cdcf67\"/\"default-dockercfg-dp8tf\""
Nov 04 02:45:19.269308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.269230    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-cdcf67\"/\"openshift-service-ca.crt\""
Nov 04 02:45:19.269308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.269266    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-cdcf67\"/\"simulator-w3-cdcf67-kserve-self-signed-certs\""
Nov 04 02:45:19.271136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.271118    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 02:45:19.346474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346442    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.346474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346477    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.346695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346507    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx8fp\" (UniqueName: \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.346742 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346688    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.346742 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346724    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.346822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.346804    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.447912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.447824    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.447912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.447859    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.447912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.447886    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.447925    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx8fp\" (UniqueName: \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.448035    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.448093    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.448285    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.448347    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.448487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.448409    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.450684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.450662    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.450837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.450734    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.508837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.508802    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx8fp\" (UniqueName: \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\") pod \"simulator-w3-cdcf67-kserve-846669b9cc-9lcdp\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") " pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:19.775686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:19.775651    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:20.182972 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:20.182951    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 02:45:20.185730 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:45:20.185702    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod140ff011_9fdc_4687_95e4_d56d141a8ac2.slice/crio-e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583 WatchSource:0}: Error finding container e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583: Status 404 returned error can't find the container with id e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583
Nov 04 02:45:20.975058 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:20.975031    6019 scope.go:122] "RemoveContainer" containerID="c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b"
Nov 04 02:45:21.080461 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:21.080427    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 02:45:21.104829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:21.104791    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" event={"ID":"140ff011-9fdc-4687-95e4-d56d141a8ac2","Type":"ContainerStarted","Data":"e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583"}
Nov 04 02:45:22.114226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.114134    6019 generic.go:356] "Generic (PLEG): container finished" podID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerID="9cbc7dd89f12ca4a328c044ecf8851cb86d5b23e38c89b41a195ed71b11a6f09" exitCode=60
Nov 04 02:45:22.114679 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.114211    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerDied","Data":"9cbc7dd89f12ca4a328c044ecf8851cb86d5b23e38c89b41a195ed71b11a6f09"}
Nov 04 02:45:22.114679 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.114324    6019 scope.go:122] "RemoveContainer" containerID="c1be718d8aa97336bcfd72362ce9ad2cced517afcbeba0d5cb4707b285f8066b"
Nov 04 02:45:22.114791 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.114715    6019 scope.go:122] "RemoveContainer" containerID="9cbc7dd89f12ca4a328c044ecf8851cb86d5b23e38c89b41a195ed71b11a6f09"
Nov 04 02:45:22.115008 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:22.114986    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj_odh-ai-gateway-infra(e24b5eeb-9d22-41b2-8962-8c4397825662)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662"
Nov 04 02:45:22.433513 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.433420    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:22.605085 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.605036    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:22.758332 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:22.758296    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:23.121996 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.121910    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" event={"ID":"140ff011-9fdc-4687-95e4-d56d141a8ac2","Type":"ContainerStarted","Data":"958b0a3dc2d5b810262c8c098469568a392e2ead565fe650e147c4a0a6c69879"}
Nov 04 02:45:23.239813 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.239757    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podStartSLOduration=2.144497698 podStartE2EDuration="4.239739689s" totalImagesPullingTime="2.095241991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:45:23.239352554 +0000 UTC m=+678.380365961" watchObservedRunningTime="2026-11-04 02:45:23.239739689 +0000 UTC m=+678.380753095"
Nov 04 02:45:23.290244 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.290218    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:23.449828 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.449804    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:23.498082 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.498053    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\" (UniqueName: \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\") pod \"e24b5eeb-9d22-41b2-8962-8c4397825662\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") "
Nov 04 02:45:23.498198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.498160    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\") pod \"e24b5eeb-9d22-41b2-8962-8c4397825662\" (UID: \"e24b5eeb-9d22-41b2-8962-8c4397825662\") "
Nov 04 02:45:23.498452 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.498424    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca" pod "e24b5eeb-9d22-41b2-8962-8c4397825662" (UID: "e24b5eeb-9d22-41b2-8962-8c4397825662"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:45:23.500355 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.500331    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99" pod "e24b5eeb-9d22-41b2-8962-8c4397825662" (UID: "e24b5eeb-9d22-41b2-8962-8c4397825662"). InnerVolumeSpecName "kube-api-access-9wl99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:45:23.598994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.598964    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wl99\" (UniqueName: \"kubernetes.io/projected/e24b5eeb-9d22-41b2-8962-8c4397825662-kube-api-access-9wl99\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:45:23.598994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.598990    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b5eeb-9d22-41b2-8962-8c4397825662-maas-api-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:45:23.776394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:23.776366    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:24.128091 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.127995    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"
Nov 04 02:45:24.128091 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.128036    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj" event={"ID":"e24b5eeb-9d22-41b2-8962-8c4397825662","Type":"ContainerDied","Data":"15bd2bbcd1a73003374fcd06f565514cbf5677e59549edef0cab4097e87d090d"}
Nov 04 02:45:24.128583 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.128124    6019 scope.go:122] "RemoveContainer" containerID="9cbc7dd89f12ca4a328c044ecf8851cb86d5b23e38c89b41a195ed71b11a6f09"
Nov 04 02:45:24.291567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.291533    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:24.297471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.297447    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-fe4e0c-29896005-hrpzj"]
Nov 04 02:45:24.792873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.792821    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:24.980946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:24.980905    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" path="/var/lib/kubelet/pods/e24b5eeb-9d22-41b2-8962-8c4397825662/volumes"
Nov 04 02:45:25.432007 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:25.431975    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:25.581135 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:25.581099    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:25.727102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:25.726985    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:27.702346 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:27.702313    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:27.867258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:27.867220    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:28.093490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:28.093436    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:29.162998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:29.162966    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:29.340033 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:29.339994    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:29.776038 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:29.776006    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:29.789036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:29.789011    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:30.171486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:30.171401    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:45:31.264306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:31.264265    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:33.185556 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:33.185523    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:36.606531 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:45:36.606492    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:45:38.203381 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:38.203345    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:43.399146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:43.399101    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:44.446257 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:44.446225    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:45.111593 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:45.111558    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:45.258482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:45.258446    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:45.401886 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:45.401803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:47.201710 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:47.201678    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:48.793323 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:48.793291    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:50.824314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:50.824283    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:51.493895 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:51.493849    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:51.642411 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:51.642381    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:54.410496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:54.410465    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:56.883019 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:56.882983    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:45:59.287005 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:45:59.286969    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:01.545733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:01.545698    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:02.241977 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:02.241946    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:02.391397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:02.391360    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:04.443517 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:04.443480    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:06.603132 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:46:06.603089    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:46:09.327432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:09.327395    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:11.789126 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:11.789090    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:11.934693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:11.934656    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:12.092863 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:12.092781    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:12.790797 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:12.790761    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:12.940957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:12.940921    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:13.086418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:13.086335    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:14.464813 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.464783    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 02:46:14.465237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465196    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.465237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465207    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.465237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465218    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.465237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465222    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.465373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465311    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.465373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.465318    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:14.468538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.468519    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477396    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477427    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477462    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5gq\" (UniqueName: \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477554    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477685 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477597    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.477685 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.477627    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.558319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.558265    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:14.570326 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.570289    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-cdcf67\"/\"premium-w3-cdcf67-kserve-self-signed-certs\""
Nov 04 02:46:14.571686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.571664    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 02:46:14.578023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.577998    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578158 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578042    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578165    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578187    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578222    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5gq\" (UniqueName: \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578334 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578270    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578479 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578458    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578545 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578516    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.578545 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.578537    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.580523 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.580503    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.580927 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.580908    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.642058 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.642023    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5gq\" (UniqueName: \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\") pod \"premium-w3-cdcf67-kserve-5c579cbdd8-dst7n\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") " pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:14.753875 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.753830    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:14.780464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:14.780422    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:15.214105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:15.214062    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 02:46:15.216796 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:46:15.216768    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2be3a14c_bdaf_4cd7_aa96_68184d60d8f2.slice/crio-0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed WatchSource:0}: Error finding container 0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed: Status 404 returned error can't find the container with id 0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed
Nov 04 02:46:15.390397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:15.390361    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" event={"ID":"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2","Type":"ContainerStarted","Data":"861e78ddd4c8e7daafdcae1cd7f234fda8f86712525804c97c24548bec139232"}
Nov 04 02:46:15.390397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:15.390401    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" event={"ID":"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2","Type":"ContainerStarted","Data":"0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed"}
Nov 04 02:46:15.507581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:15.507536    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podStartSLOduration=1.5075232029999999 podStartE2EDuration="1.507523203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:46:15.50740989 +0000 UTC m=+730.648423297" watchObservedRunningTime="2026-11-04 02:46:15.507523203 +0000 UTC m=+730.648536654"
Nov 04 02:46:15.781293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:15.781214    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:17.489146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:17.489110    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:18.725598 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:18.725560    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:19.783714 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:19.783666    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:20.862502 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:20.862446    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:24.687544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:24.687514    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:24.781497 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:24.781462    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:24.793971 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:24.793948    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:25.450388 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:25.450361    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:46:29.515156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:29.515113    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:29.659228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:29.659199    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:31.643772 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.643735    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:31.811212 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.811177    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 02:46:31.811608 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.811595    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:31.811652 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.811609    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:31.811690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.811684    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e24b5eeb-9d22-41b2-8962-8c4397825662" containerName="cleanup"
Nov 04 02:46:31.814833 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.814812    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.911102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.911009    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:31.916470 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.916431    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-e516ff\"/\"openshift-service-ca.crt\""
Nov 04 02:46:31.916661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.916541    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-e516ff\"/\"simulator-w1-e516ff-kserve-self-signed-certs\""
Nov 04 02:46:31.916661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.916558    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-e516ff\"/\"kube-root-ca.crt\""
Nov 04 02:46:31.916661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.916616    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-e516ff\"/\"default-dockercfg-9d9b9\""
Nov 04 02:46:31.918159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.918140    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 02:46:31.949115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949090    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtftm\" (UniqueName: \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.949253 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949127    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.949253 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949147    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.949337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949286    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.949337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949324    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:31.949434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:31.949358    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050653 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050615    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050827 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050709    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtftm\" (UniqueName: \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050827 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050752    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050827 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050779    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050999 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050876    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.050999 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.050916    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.051134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.051055    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.051194 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.051181    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.051303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.051282    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.053410 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.053383    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.053871 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.053841    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.114673 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.114642    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtftm\" (UniqueName: \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\") pod \"simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") " pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.127533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.127514    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:32.513909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:32.513881    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 02:46:32.517224 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:46:32.517199    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09644268_19f7_48c9_9542_f80708a833d8.slice/crio-8d595d58369a4a5d739fd36ce4f94035bd49108b59be59843c4c3c6022dc5626 WatchSource:0}: Error finding container 8d595d58369a4a5d739fd36ce4f94035bd49108b59be59843c4c3c6022dc5626: Status 404 returned error can't find the container with id 8d595d58369a4a5d739fd36ce4f94035bd49108b59be59843c4c3c6022dc5626
Nov 04 02:46:33.479447 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:33.479407    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" event={"ID":"09644268-19f7-48c9-9542-f80708a833d8","Type":"ContainerStarted","Data":"0a82e35d3a56b532fbeac9698f8c814e0ec123760953de7b426cf407a836bd21"}
Nov 04 02:46:33.479834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:33.479461    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" event={"ID":"09644268-19f7-48c9-9542-f80708a833d8","Type":"ContainerStarted","Data":"8d595d58369a4a5d739fd36ce4f94035bd49108b59be59843c4c3c6022dc5626"}
Nov 04 02:46:33.596245 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:33.596187    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podStartSLOduration=2.596169888 podStartE2EDuration="2.596169888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:46:33.595649893 +0000 UTC m=+748.736663300" watchObservedRunningTime="2026-11-04 02:46:33.596169888 +0000 UTC m=+748.737183308"
Nov 04 02:46:34.128212 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:34.128180    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:34.486484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:34.486398    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:36.602427 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:46:36.602394    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:46:39.389699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:39.389667    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:42.128249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:42.128213    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:42.140973 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:42.140949    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:42.532841 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:42.532819    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:46:44.285637 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:44.285606    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:44.600116 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:44.600001    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:49.205718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:49.205676    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:54.201124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:54.201090    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:46:59.106399 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:46:59.106362    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:00.342324 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.342294    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 02:47:00.346303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.346282    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.398702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.398669    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ebb7fd\"/\"openshift-service-ca.crt\""
Nov 04 02:47:00.425695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425664    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.425853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425712    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.425853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425734    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.425853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425838    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.425982 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425888    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gt8m\" (UniqueName: \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.425982 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.425938    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.435718 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.435691    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:00.448282 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.448254    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ebb7fd\"/\"simulator-w6-ebb7fd-kserve-self-signed-certs\""
Nov 04 02:47:00.448394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.448361    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ebb7fd\"/\"default-dockercfg-w24cn\""
Nov 04 02:47:00.448462 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.448423    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ebb7fd\"/\"kube-root-ca.crt\""
Nov 04 02:47:00.449979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.449958    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 02:47:00.526494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526452    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526503    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gt8m\" (UniqueName: \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526633    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526799 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526779    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526859 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526844    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526883    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.526949 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.526908    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.527134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.527117    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.527221 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.527200    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.529023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.529002    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.529507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.529475    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.598224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.598152    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gt8m\" (UniqueName: \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\") pod \"simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") " pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:00.660271 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:00.660231    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:01.040502 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:01.040476    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 02:47:01.043025 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:47:01.043000    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b5bc6a6_9bb7_4535_9952_e6ffdac4f6d5.slice/crio-14b308bad74c0290a2fe329d463d444f7bc902a754028f243b26be795fb76765 WatchSource:0}: Error finding container 14b308bad74c0290a2fe329d463d444f7bc902a754028f243b26be795fb76765: Status 404 returned error can't find the container with id 14b308bad74c0290a2fe329d463d444f7bc902a754028f243b26be795fb76765
Nov 04 02:47:01.621508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:01.621468    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" event={"ID":"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5","Type":"ContainerStarted","Data":"ad5cd9c0429c606e3b5bb227700fd87e3a11c8d38553ff3d754c2a0ac691d8b8"}
Nov 04 02:47:01.621508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:01.621510    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" event={"ID":"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5","Type":"ContainerStarted","Data":"14b308bad74c0290a2fe329d463d444f7bc902a754028f243b26be795fb76765"}
Nov 04 02:47:01.660686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:01.660660    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:01.740390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:01.740337    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podStartSLOduration=1.740324288 podStartE2EDuration="1.740324288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:47:01.73887248 +0000 UTC m=+776.879885887" watchObservedRunningTime="2026-11-04 02:47:01.740324288 +0000 UTC m=+776.881337694"
Nov 04 02:47:03.998325 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:03.998294    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:06.603326 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:47:06.603289    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:47:08.993098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:08.993035    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:09.574230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:09.574196    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:10.661143 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:10.661107    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:10.674820 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:10.674795    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:11.672748 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:11.672717    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:47:13.877329 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:13.877291    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:16.295198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:16.295163    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:16.442891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:16.442854    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:18.451326 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:18.451289    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:18.775538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:18.775495    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:23.676613 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:23.676575    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:28.577365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:28.577327    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:29.872956 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:29.872919    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:31.997627 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:31.997598    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:33.485758 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.485723    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:33.791211 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.791170    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 02:47:33.795109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.795088    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.847159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.847131    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-3b7476\"/\"openshift-service-ca.crt\""
Nov 04 02:47:33.847308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.847281    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-3b7476\"/\"kube-root-ca.crt\""
Nov 04 02:47:33.876287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876260    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.876427 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876325    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.876427 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876395    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.876498 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876426    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.876540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876518    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.876584 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.876571    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plxsf\" (UniqueName: \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.879629 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.879610    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:33.896875 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.896856    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b7476\"/\"default-dockercfg-q7bsj\""
Nov 04 02:47:33.896974 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.896952    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b7476\"/\"premium-w5-3b7476-kserve-self-signed-certs\""
Nov 04 02:47:33.898604 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.898587    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 02:47:33.977413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977378    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977432    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977466    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977496    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977534    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977551    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plxsf\" (UniqueName: \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977788    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977861    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.977981 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.977927    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.980021 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.979993    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:33.980290 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:33.980274    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:34.038773 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.038735    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plxsf\" (UniqueName: \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\") pod \"premium-w5-3b7476-kserve-56cfb88bfb-965k8\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") " pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:34.107732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.107655    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:34.513405 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.513376    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 02:47:34.516004 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:47:34.515980    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb70d340e_cc4c_4e8d_8c9d_f4660894d103.slice/crio-a713eab3628ff6750d750ec0a71a111e82b8c4d3056b82749b5de32cc1d5cddf WatchSource:0}: Error finding container a713eab3628ff6750d750ec0a71a111e82b8c4d3056b82749b5de32cc1d5cddf: Status 404 returned error can't find the container with id a713eab3628ff6750d750ec0a71a111e82b8c4d3056b82749b5de32cc1d5cddf
Nov 04 02:47:34.807544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.807459    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" event={"ID":"b70d340e-cc4c-4e8d-8c9d-f4660894d103","Type":"ContainerStarted","Data":"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"}
Nov 04 02:47:34.807544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.807498    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" event={"ID":"b70d340e-cc4c-4e8d-8c9d-f4660894d103","Type":"ContainerStarted","Data":"a713eab3628ff6750d750ec0a71a111e82b8c4d3056b82749b5de32cc1d5cddf"}
Nov 04 02:47:34.924311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:34.924264    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podStartSLOduration=1.924251136 podStartE2EDuration="1.924251136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:47:34.923217007 +0000 UTC m=+810.064230421" watchObservedRunningTime="2026-11-04 02:47:34.924251136 +0000 UTC m=+810.065264556"
Nov 04 02:47:35.108099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:35.108008    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:36.602686 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:47:36.602650    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:47:38.396132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:38.396096    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:43.283957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:43.283924    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:44.108034 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:44.108001    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:44.120438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:44.120412    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:44.873176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:44.873148    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:47:45.697472 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:45.697434    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:45.842672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:45.842644    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:48.182044 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:48.182009    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:53.120600 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:53.120558    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:54.113592 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:54.113559    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:54.256466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:54.256432    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:47:57.992415 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:47:57.992379    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:02.001530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:02.001474    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:02.922018 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:02.921974    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:06.602630 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:48:06.602594    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:48:07.790105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:07.790046    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:10.506474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:10.506384    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:10.654241 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:10.654206    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:12.680138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:12.680100    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:17.182192 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.182156    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 02:48:17.185675 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.185659    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.238172 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.238138    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-203c35\"/\"kube-root-ca.crt\""
Nov 04 02:48:17.275369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.275321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:17.287796 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.287772    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-203c35\"/\"openshift-service-ca.crt\""
Nov 04 02:48:17.287796 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.287792    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-203c35\"/\"premium-w4-203c35-kserve-self-signed-certs\""
Nov 04 02:48:17.287952 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.287798    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-203c35\"/\"default-dockercfg-tkts4\""
Nov 04 02:48:17.289658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.289642    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 02:48:17.349710 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.349675    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.350046 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.350020    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.350273 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.350243    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.350400 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.350331    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.350400 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.350371    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.350529 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.350471    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9nj\" (UniqueName: \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451492    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451534    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451566    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9nj\" (UniqueName: \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451621    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451674    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.451857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451732    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.452025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.451963    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.452102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.452059    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.452572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.452406    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.454496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.454474    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.454725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.454705    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.515662 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.515625    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9nj\" (UniqueName: \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\") pod \"premium-w4-203c35-kserve-dc5c4d6c4-vhvjg\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") " pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:17.777237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.777193    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:17.797233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:17.797211    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:18.187084 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:18.187049    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 02:48:18.189960 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:48:18.189929    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f43d768_d8d6_4b1f_9b23_8e1d4652993a.slice/crio-439dad69968cf50895d4ec991a75113c0d871291c017f821e0645a6cace556e5 WatchSource:0}: Error finding container 439dad69968cf50895d4ec991a75113c0d871291c017f821e0645a6cace556e5: Status 404 returned error can't find the container with id 439dad69968cf50895d4ec991a75113c0d871291c017f821e0645a6cace556e5
Nov 04 02:48:19.043464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:19.043432    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" event={"ID":"2f43d768-d8d6-4b1f-9b23-8e1d4652993a","Type":"ContainerStarted","Data":"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"}
Nov 04 02:48:19.043464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:19.043470    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" event={"ID":"2f43d768-d8d6-4b1f-9b23-8e1d4652993a","Type":"ContainerStarted","Data":"439dad69968cf50895d4ec991a75113c0d871291c017f821e0645a6cace556e5"}
Nov 04 02:48:19.161189 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:19.161136    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podStartSLOduration=2.16112099 podStartE2EDuration="2.16112099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:48:19.159386452 +0000 UTC m=+854.300399860" watchObservedRunningTime="2026-11-04 02:48:19.16112099 +0000 UTC m=+854.302134397"
Nov 04 02:48:19.798307 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:19.798278    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:22.678454 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:22.678407    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:26.484086 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.484039    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 02:48:26.487806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.487785    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.552872 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.552840    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:26.585101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.584461    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 02:48:26.589344 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.589322    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ebb7fd\"/\"premium-w6-ebb7fd-kserve-self-signed-certs\""
Nov 04 02:48:26.651154 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651126    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.651154 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651157    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.651388 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651182    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.651388 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651351    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7gpj\" (UniqueName: \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.651494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651453    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.651548 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.651521    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.752990 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.752962    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753028    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gpj\" (UniqueName: \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753093    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753124    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753303    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753341    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753518 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753487    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753668 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753637    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.753735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.753701    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.756199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.756170    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.756575 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.756555    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:26.814726 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:26.814690    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7gpj\" (UniqueName: \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\") pod \"premium-w6-ebb7fd-kserve-75dd4dc966-wghjv\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") " pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:27.103331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:27.103247    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:27.476981 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:27.476958    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 02:48:27.479026 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:48:27.478997    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ff179f5_d5b5_4c2c_875e_28713181605f.slice/crio-1fdce140cc3ce40fa0b7e7f7430f47758a1ddfdba6deeafe86243e64846746c9 WatchSource:0}: Error finding container 1fdce140cc3ce40fa0b7e7f7430f47758a1ddfdba6deeafe86243e64846746c9: Status 404 returned error can't find the container with id 1fdce140cc3ce40fa0b7e7f7430f47758a1ddfdba6deeafe86243e64846746c9
Nov 04 02:48:27.601240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:27.601205    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:27.797954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:27.797923    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:27.810009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:27.809977    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:28.089443 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:28.089349    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" event={"ID":"7ff179f5-d5b5-4c2c-875e-28713181605f","Type":"ContainerStarted","Data":"debc8e129cf4ecce991a21425d1942595f0dad2b0c19b7cc95b4230d3e364761"}
Nov 04 02:48:28.089443 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:28.089400    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" event={"ID":"7ff179f5-d5b5-4c2c-875e-28713181605f","Type":"ContainerStarted","Data":"1fdce140cc3ce40fa0b7e7f7430f47758a1ddfdba6deeafe86243e64846746c9"}
Nov 04 02:48:28.100872 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:28.100849    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:48:28.104029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:28.104012    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:28.206727 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:28.206677    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podStartSLOduration=2.206662368 podStartE2EDuration="2.206662368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:48:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:48:28.205000448 +0000 UTC m=+863.346013872" watchObservedRunningTime="2026-11-04 02:48:28.206662368 +0000 UTC m=+863.347675775"
Nov 04 02:48:30.228122 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:30.228066    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:30.395409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:30.395375    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:32.345539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:32.345500    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:32.488311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:32.488279    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:36.602402 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:48:36.602356    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:48:37.104246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:37.104216    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:37.116865 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:37.116843    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:37.147120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:37.147065    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:48:37.417616 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:37.417540    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:42.468629 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:42.468586    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:42.618208 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:42.618168    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:46.373607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.373571    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 02:48:46.377298 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.377279    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.429472 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.429451    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-cdcf67-openshift-default-dockercfg-pbxqp\""
Nov 04 02:48:46.454009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.453982    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454015    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454057    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454171    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454206    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smhc\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454234    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454411 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454386    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454432    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.454522 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.454498    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.481671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.481640    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 02:48:46.555407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555375    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555412    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555458    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555488    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555522    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9smhc\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555563    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555612    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555638    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555665    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555941 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555824    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.555941 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555889    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.556026 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.555987    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.556124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.556107    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.556705 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.556680    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.558028 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.557998    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.558165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.558150    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.616407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.616375    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.616571 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.616465    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smhc\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:46.690615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:46.690550    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:47.119404 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.119344    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 02:48:47.121762 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:48:47.121735    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ac5484_4613_4022_984a_d34327ae5661.slice/crio-d85bdabd13e05109c47ed4db0840d1e45c3af0d8e40da6a8c823ef6efeee514f WatchSource:0}: Error finding container d85bdabd13e05109c47ed4db0840d1e45c3af0d8e40da6a8c823ef6efeee514f: Status 404 returned error can't find the container with id d85bdabd13e05109c47ed4db0840d1e45c3af0d8e40da6a8c823ef6efeee514f
Nov 04 02:48:47.124173 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.124144    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:48:47.124226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.124211    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:48:47.124263 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.124240    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:48:47.189419 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.189388    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" event={"ID":"97ac5484-4613-4022-984a-d34327ae5661","Type":"ContainerStarted","Data":"d85bdabd13e05109c47ed4db0840d1e45c3af0d8e40da6a8c823ef6efeee514f"}
Nov 04 02:48:47.383693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:47.383616    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:48.195988 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:48.195952    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" event={"ID":"97ac5484-4613-4022-984a-d34327ae5661","Type":"ContainerStarted","Data":"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"}
Nov 04 02:48:48.430722 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:48.430668    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podStartSLOduration=2.430653547 podStartE2EDuration="2.430653547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:48:48.379310575 +0000 UTC m=+883.520323984" watchObservedRunningTime="2026-11-04 02:48:48.430653547 +0000 UTC m=+883.571667025"
Nov 04 02:48:48.691639 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:48.691609    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:48.692933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:48.692913    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:48:48.692988 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:48.692958    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:48:49.691453 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:49.691418    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:48:49.691851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:49.691480    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:48:50.691267 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:50.691236    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:48:50.691454 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:50.691286    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:48:51.691669 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:51.691638    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:48:51.692096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:51.691697    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:48:52.403868 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:52.403838    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:48:52.691832 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:52.691740    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:48:52.691832 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:52.691796    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:48:53.831580 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:53.831546    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:53.832085 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:53.832005    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:53.832899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:53.832876    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 02:48:57.368206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:48:57.368167    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:02.303935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:02.303901    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:05.070910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:05.070886    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:49:05.103233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:05.103208    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:49:06.603635 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:49:06.603598    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:49:07.322980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:07.322942    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:12.331920 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:12.331888    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:17.179575 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:17.179518    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:22.090183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:22.090150    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:27.025865 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:27.025830    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:30.639337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:30.639301    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:32.083229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.083194    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:32.365236 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.365152    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 02:49:32.369886 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.369861    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.422143 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.422120    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-e516ff-openshift-default-dockercfg-5x2gt\""
Nov 04 02:49:32.458781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.458754    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.458915 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.458791    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.458915 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.458845    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.458915 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.458891    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.459031 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.458982    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.459031 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.459000    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.459031 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.459016    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.459149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.459033    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntvhz\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.459182 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.459175    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.473958 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.473935    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 02:49:32.559943 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.559905    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.559943 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.559946    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.560219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560033    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.560219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560056    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.560219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560129    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.560219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560155    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.560561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560536    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560974    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntvhz\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560595    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561197 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560670    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561197 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560888    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561197 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.560542    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.561206    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.561424 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.561358    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.562612 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.562591    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.562775 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.562757    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.622279 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.622212    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.622392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.622321    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntvhz\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.682766 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.682738    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:32.707228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:32.707203    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:33.109331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.109304    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 02:49:33.111847 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:49:33.111822    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2042e3c_08cb_4694_92ed_e22b87d310cc.slice/crio-bc87fe1d391c430de90e94aad8ccd9fb1b7b5a1f8e60a63a54ea63276355fa56 WatchSource:0}: Error finding container bc87fe1d391c430de90e94aad8ccd9fb1b7b5a1f8e60a63a54ea63276355fa56: Status 404 returned error can't find the container with id bc87fe1d391c430de90e94aad8ccd9fb1b7b5a1f8e60a63a54ea63276355fa56
Nov 04 02:49:33.114210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.114177    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:49:33.114293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.114249    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:49:33.114293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.114277    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:49:33.433210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.433108    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" event={"ID":"b2042e3c-08cb-4694-92ed-e22b87d310cc","Type":"ContainerStarted","Data":"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"}
Nov 04 02:49:33.433210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.433173    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" event={"ID":"b2042e3c-08cb-4694-92ed-e22b87d310cc","Type":"ContainerStarted","Data":"bc87fe1d391c430de90e94aad8ccd9fb1b7b5a1f8e60a63a54ea63276355fa56"}
Nov 04 02:49:33.555942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.555890    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podStartSLOduration=1.555876396 podStartE2EDuration="1.555876396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:49:33.554030382 +0000 UTC m=+928.695043791" watchObservedRunningTime="2026-11-04 02:49:33.555876396 +0000 UTC m=+928.696889802"
Nov 04 02:49:33.684480 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:33.684403    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:34.685757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:34.685724    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:34.686179 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:34.685776    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:35.683342 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:35.683314    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:35.683535 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:35.683360    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:36.604253 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:49:36.604214    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:49:36.684681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:36.684652    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:36.684834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:36.684698    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:37.074672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:37.074644    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:37.683409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:37.683372    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:37.683793 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:37.683428    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:38.683588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:38.683549    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:38.683966 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:38.683606    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:39.683824 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:39.683750    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:49:39.684218 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:39.683819    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:49:40.687738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:40.687703    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:40.688185 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:40.687965    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:40.688540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:40.688524    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 02:49:41.999699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:41.999669    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:45.623572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:45.623534    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:46.876695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:46.876661    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:52.087836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:52.087803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:56.845693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:56.845661    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:57.099955 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:57.099865    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:49:57.242838 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:49:57.242800    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:01.694509 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.694479    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 02:50:01.702965 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.698720    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.752162 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.752139    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-a0bcbc-openshift-default-dockercfg-z8ns2\""
Nov 04 02:50:01.779592 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779568    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779607    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779627    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779805 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779728    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779805 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779754    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779867 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779824    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx4l5\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779867 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779854    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779931 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779876    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.779964 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.779930    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.803770 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.803746    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 02:50:01.881105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881059    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881128    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881149    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881169    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx4l5\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881198    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881233    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881266    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881416    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881467    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.881725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881554    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.882136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881750    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.882136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.881770    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.882238 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.882176    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.882296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.882260    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.883857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.883838    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.884263 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.884246    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.944420 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.944390    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:01.944626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:01.944582    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx4l5\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:02.005578 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.005543    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:02.015586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.015546    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:02.467735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.467715    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 02:50:02.470406 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:50:02.470376    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd5be74_a0c4_4baa_a3c9_ea620584601c.slice/crio-51acfa8ded4f8714464b185a65809adc62aca93cd80fad3010e878e4824fc5e9 WatchSource:0}: Error finding container 51acfa8ded4f8714464b185a65809adc62aca93cd80fad3010e878e4824fc5e9: Status 404 returned error can't find the container with id 51acfa8ded4f8714464b185a65809adc62aca93cd80fad3010e878e4824fc5e9
Nov 04 02:50:02.472708 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.472680    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:02.472772 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.472741    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:02.472807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.472772    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:02.634677 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.634643    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" event={"ID":"3bd5be74-a0c4-4baa-a3c9-ea620584601c","Type":"ContainerStarted","Data":"3599b89c92730ee25554828909ba25f02cea9e603570edc1a55b69bd15c9577a"}
Nov 04 02:50:02.634864 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.634702    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" event={"ID":"3bd5be74-a0c4-4baa-a3c9-ea620584601c","Type":"ContainerStarted","Data":"51acfa8ded4f8714464b185a65809adc62aca93cd80fad3010e878e4824fc5e9"}
Nov 04 02:50:02.756885 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:02.756835    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podStartSLOduration=1.7568153130000002 podStartE2EDuration="1.756815313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:50:02.754489454 +0000 UTC m=+957.895502862" watchObservedRunningTime="2026-11-04 02:50:02.756815313 +0000 UTC m=+957.897828720"
Nov 04 02:50:03.016690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:03.016605    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:03.018278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:03.018256    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:03.018388 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:03.018299    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:04.016757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:04.016720    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:04.017178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:04.016778    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:05.016330 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:05.016293    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:05.016550 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:05.016344    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:06.016254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:06.016221    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:06.016656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:06.016273    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:06.617089 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:50:06.616855    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:50:07.016684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:07.016645    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:07.017203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:07.016734    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:07.110636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:07.110540    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:08.026098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:08.025472    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:08.026098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:08.025548    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:09.020094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:09.018481    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 04 02:50:09.020094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:09.018636    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 04 02:50:11.060247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:11.060206    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:50:11.060829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:11.060796    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:50:11.134100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:11.134049    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:11.135345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:11.135314    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:11.136228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:11.136203    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 02:50:12.214716 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:12.214678    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:17.088101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.088045    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 02:50:17.091801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.091778    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.115994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.115970    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:17.165669 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.165617    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 02:50:17.187462 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187432    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187524    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187549    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187574    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187598    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187613    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187632    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187674    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnncw\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.187786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.187716    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288706    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnncw\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288760    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288808    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288868    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288886    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.288905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288903    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288926    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288950    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.288977    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289383 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.289330    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.289385    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.289399    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.289652    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.289734 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.289715    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.291530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.291506    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.291757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.291738    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.349746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.349669    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnncw\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.351418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.351391    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.366768 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.366742    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 02:50:17.371865 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.371846    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.405483 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.405460    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:17.423896 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.423872    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-fe4e0c-openshift-default-dockercfg-gx9rt\""
Nov 04 02:50:17.436304 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.436272    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 02:50:17.491494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491461    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491534    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491619    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491756 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491662    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491756 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491689    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491805    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.491900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.491884    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.492024 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.492004    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.492084 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.492032    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb6d8\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593228 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593184    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593260    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593393 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593369    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593392    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb6d8\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593427    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593479    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593506    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593539    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593561    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593689    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593851    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.593946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.593935    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.594217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.594192    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.594338 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.594204    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.595649 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.595633    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.596140 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.596119    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.656435 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.656365    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb6d8\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.656435 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.656388    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.685412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.685384    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:17.844280 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.844097    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 02:50:17.846651 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:50:17.846621    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6288f05_af04_46c0_bb21_71a943669694.slice/crio-6e2e59528e1e176155146c207782bca7827b571d01d89a89f32f9bfdf6e9b759 WatchSource:0}: Error finding container 6e2e59528e1e176155146c207782bca7827b571d01d89a89f32f9bfdf6e9b759: Status 404 returned error can't find the container with id 6e2e59528e1e176155146c207782bca7827b571d01d89a89f32f9bfdf6e9b759
Nov 04 02:50:17.849331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.849301    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:17.849420 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.849363    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:17.849420 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:17.849392    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.032989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.032948    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 02:50:18.037566 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.037543    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.070293 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.069766    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 02:50:18.071879 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:50:18.071851    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8ab58a_05e8_4c31_97b8_726adf35bd57.slice/crio-2d7da5bc6cde6cbc005a4a6016ad4def83ab272c3fbcd7b3152f719603f4c088 WatchSource:0}: Error finding container 2d7da5bc6cde6cbc005a4a6016ad4def83ab272c3fbcd7b3152f719603f4c088: Status 404 returned error can't find the container with id 2d7da5bc6cde6cbc005a4a6016ad4def83ab272c3fbcd7b3152f719603f4c088
Nov 04 02:50:18.074451 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.074407    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.074548 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.074481    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.074548 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.074507    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.096378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.090272    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-203c35-openshift-default-dockercfg-ggkdg\""
Nov 04 02:50:18.098165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.098139    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 02:50:18.103636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.103611    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.103774 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.103691    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.104563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.103932    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.104748 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.104731    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.104892 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.104879    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.105047 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.105027    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.105246 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.105108    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.105313 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.105243    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg7fz\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.105313 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.105275    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.206705 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206666    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.206902 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206725    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.206902 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206780    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.206902 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206831    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.207119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206903    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg7fz\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.207119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.206938    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.207119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.207014    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.207119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.207055    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.207316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.207156    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.209296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.208926    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.209296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.209057    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.209296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.209126    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.209296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.209182    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.209582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.209535    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.211725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.211704    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.212353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.212331    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.271626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.271592    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg7fz\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.271626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.271612    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.351921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.351800    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:18.772095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.769851    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 02:50:18.802661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.802259    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" event={"ID":"eb8ab58a-05e8-4c31-97b8-726adf35bd57","Type":"ContainerStarted","Data":"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"}
Nov 04 02:50:18.802661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.802319    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" event={"ID":"eb8ab58a-05e8-4c31-97b8-726adf35bd57","Type":"ContainerStarted","Data":"2d7da5bc6cde6cbc005a4a6016ad4def83ab272c3fbcd7b3152f719603f4c088"}
Nov 04 02:50:18.804738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.804702    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" event={"ID":"b6288f05-af04-46c0-bb21-71a943669694","Type":"ContainerStarted","Data":"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"}
Nov 04 02:50:18.804848 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.804766    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" event={"ID":"b6288f05-af04-46c0-bb21-71a943669694","Type":"ContainerStarted","Data":"6e2e59528e1e176155146c207782bca7827b571d01d89a89f32f9bfdf6e9b759"}
Nov 04 02:50:18.806992 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:50:18.806967    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfda9917_4319_4230_89a0_7624e0236375.slice/crio-1f706acada5d2eece4ee20f5cf8a2dac2864f4af70d68d57a7f41614ce079612 WatchSource:0}: Error finding container 1f706acada5d2eece4ee20f5cf8a2dac2864f4af70d68d57a7f41614ce079612: Status 404 returned error can't find the container with id 1f706acada5d2eece4ee20f5cf8a2dac2864f4af70d68d57a7f41614ce079612
Nov 04 02:50:18.809686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.809659    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.809750 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.809721    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.809786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.809751    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:50:18.926140 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:18.926061    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podStartSLOduration=1.926044649 podStartE2EDuration="1.926044649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:50:18.923148952 +0000 UTC m=+974.064162360" watchObservedRunningTime="2026-11-04 02:50:18.926044649 +0000 UTC m=+974.067058059"
Nov 04 02:50:19.406556 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.406517    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:19.408130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.408101    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:19.408225 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.408157    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:19.695366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.695274    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:19.695701 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.695681    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:19.695837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.695814    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:19.832372 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.831369    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" event={"ID":"cfda9917-4319-4230-89a0-7624e0236375","Type":"ContainerStarted","Data":"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"}
Nov 04 02:50:19.832372 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.831446    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" event={"ID":"cfda9917-4319-4230-89a0-7624e0236375","Type":"ContainerStarted","Data":"1f706acada5d2eece4ee20f5cf8a2dac2864f4af70d68d57a7f41614ce079612"}
Nov 04 02:50:19.965157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:19.963241    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podStartSLOduration=2.963221848 podStartE2EDuration="2.963221848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:50:19.078683998 +0000 UTC m=+974.219697407" watchObservedRunningTime="2026-11-04 02:50:19.963221848 +0000 UTC m=+975.104235256"
Nov 04 02:50:20.379099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.369936    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:20.379099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.370013    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:20.379099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.370126    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:20.415098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.414520    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:20.415098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.414579    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:20.689549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.689460    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:20.689784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:20.689756    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:21.362098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.358460    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:21.362098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.358522    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:21.415098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.411594    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:21.415098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.411655    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:21.695613 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.695513    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:21.695890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:21.695855    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:22.126693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.126563    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:22.361099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.356453    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:22.361099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.356512    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:22.408609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.408517    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:22.408855 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.408827    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:22.692159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.685767    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:22.692159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:22.685827    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:23.364099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.361461    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:23.364099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.361527    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:23.411097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.407437    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:23.411097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.407496    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:23.696157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.688583    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:23.696157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:23.688662    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:24.355094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.354461    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:24.355094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.354539    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:24.406732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.406691    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 02:50:24.406916 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.406773    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 02:50:24.686397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.686312    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 04 02:50:24.686397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:24.686375    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 04 02:50:25.352681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.352650    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 02:50:25.353059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.352705    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 02:50:25.410705 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.410676    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:25.411044 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.411021    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:25.411914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.411894    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 02:50:25.567101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.567025    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podStartSLOduration=8.567009757 podStartE2EDuration="8.567009757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:50:19.9654816 +0000 UTC m=+975.106495008" watchObservedRunningTime="2026-11-04 02:50:25.567009757 +0000 UTC m=+980.708023164"
Nov 04 02:50:25.691518 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.691445    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:25.691873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.691856    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:25.692814 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:25.692791    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 02:50:26.359447 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:26.359408    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:26.360716 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:26.360643    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:26.361448 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:26.361428    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 02:50:27.137989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:27.137956    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:30.935496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:30.935460    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:31.992275 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:31.992240    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:33.050431 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:33.050399    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:33.597389 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:33.597357    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:50:33.597600 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:33.597577    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy" containerID="cri-o://049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f" gracePeriod=30
Nov 04 02:50:36.602946 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:50:36.602907    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:50:37.032945 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:37.032911    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:38.862803 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.862740    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:50:38.943429 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943397    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943461    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943649 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943585    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943649 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943615    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943652    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943683    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:50:38.943749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943732    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943760    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.943900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943833    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.944060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943865    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:50:38.944060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943870    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:50:38.944060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943943    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:50:38.944060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.943873    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\") pod \"106f0b36-091f-49d8-af2d-e8e46d131645\" (UID: \"106f0b36-091f-49d8-af2d-e8e46d131645\") "
Nov 04 02:50:38.944300 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944131    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:50:38.944785 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944760    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:38.944897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944791    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:38.944897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944808    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:38.944897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944823    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:38.944897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.944837    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/106f0b36-091f-49d8-af2d-e8e46d131645-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:38.946000 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.945974    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:50:38.946199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.946182    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:50:38.946541 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.946520    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "kube-api-access-88d4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:50:38.946625 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.946536    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token" pod "106f0b36-091f-49d8-af2d-e8e46d131645" (UID: "106f0b36-091f-49d8-af2d-e8e46d131645"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:50:38.987638 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.987610    6019 generic.go:356] "Generic (PLEG): container finished" podID="106f0b36-091f-49d8-af2d-e8e46d131645" containerID="049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f" exitCode=0
Nov 04 02:50:38.987765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.987648    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" event={"ID":"106f0b36-091f-49d8-af2d-e8e46d131645","Type":"ContainerDied","Data":"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"}
Nov 04 02:50:38.987765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.987666    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"
Nov 04 02:50:38.987765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.987689    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk" event={"ID":"106f0b36-091f-49d8-af2d-e8e46d131645","Type":"ContainerDied","Data":"798f679718e2a9e455a8562d31af97305268f848c171e8d983ced9e71fd5b7c3"}
Nov 04 02:50:38.987765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.987707    6019 scope.go:122] "RemoveContainer" containerID="049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"
Nov 04 02:50:38.997060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.997046    6019 scope.go:122] "RemoveContainer" containerID="049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"
Nov 04 02:50:38.997369 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:50:38.997349    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f\": container with ID starting with 049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f not found: ID does not exist" containerID="049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"
Nov 04 02:50:38.997434 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:38.997382    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f"} err="failed to get container status \"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f\": rpc error: code = NotFound desc = could not find container \"049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f\": container with ID starting with 049afaa10cc6d884d293bb6001b05554af3047727d14e51a73cc90a37a12f16f not found: ID does not exist"
Nov 04 02:50:39.045646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.045624    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/106f0b36-091f-49d8-af2d-e8e46d131645-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:39.045646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.045646    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88d4h\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-kube-api-access-88d4h\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:39.045792 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.045657    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/106f0b36-091f-49d8-af2d-e8e46d131645-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:39.045792 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.045665    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/106f0b36-091f-49d8-af2d-e8e46d131645-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:50:39.145051 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.144976    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:50:39.149915 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:39.149885    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk"]
Nov 04 02:50:40.981847 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:40.981815    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" path="/var/lib/kubelet/pods/106f0b36-091f-49d8-af2d-e8e46d131645/volumes"
Nov 04 02:50:42.019720 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:42.019683    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:46.916933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:46.916901    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:47.697985 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:47.697953    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:51.919199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:51.919163    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:56.873347 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:56.873306    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:50:57.687347 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:50:57.687316    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:01.884544 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:01.884506    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:03.705484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.705452    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 02:51:03.706050 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.706031    6019 cpu_manager.go:414] "containerMap: removing container" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy"
Nov 04 02:51:03.706124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.706052    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy"
Nov 04 02:51:03.706160 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.706155    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="106f0b36-091f-49d8-af2d-e8e46d131645" containerName="istio-proxy"
Nov 04 02:51:03.710642 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.710624    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.763807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.763770    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-3b7476-openshift-default-dockercfg-t2888\""
Nov 04 02:51:03.817968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.817932    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 02:51:03.849687 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.849649    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.849863 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.849772    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6slbt\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.849863 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.849799    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.849863 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.849841    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.850029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.850011    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.850087 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.850044    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.850172 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.850146    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.850224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.850203    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.850312 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.850294    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.951819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.951783    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.951819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.951822    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.951845    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.951960    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952015    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952109    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952172    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6slbt\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952201    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952253    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952438    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952479    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952632    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952768 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952630    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.952768 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.952632    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.954689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.954668    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:03.954847 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:03.954831    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:04.013355 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.013322    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:04.013507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.013438    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6slbt\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:04.024394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.024372    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:04.452242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.452214    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 02:51:04.454712 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:51:04.454676    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa9edf1_0bca_4cc2_b8db_2420bedcad59.slice/crio-048a85463eb1b6a5bb25b4974d2841b05ee7205665bef519eb6cc64d6499c7c9 WatchSource:0}: Error finding container 048a85463eb1b6a5bb25b4974d2841b05ee7205665bef519eb6cc64d6499c7c9: Status 404 returned error can't find the container with id 048a85463eb1b6a5bb25b4974d2841b05ee7205665bef519eb6cc64d6499c7c9
Nov 04 02:51:04.457141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.457107    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:04.457234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.457179    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:04.457234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:04.457210    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:05.108790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:05.108756    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:05.247936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:05.247896    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" event={"ID":"bfa9edf1-0bca-4cc2-b8db-2420bedcad59","Type":"ContainerStarted","Data":"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"}
Nov 04 02:51:05.248146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:05.247995    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" event={"ID":"bfa9edf1-0bca-4cc2-b8db-2420bedcad59","Type":"ContainerStarted","Data":"048a85463eb1b6a5bb25b4974d2841b05ee7205665bef519eb6cc64d6499c7c9"}
Nov 04 02:51:05.375117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:05.374977    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podStartSLOduration=2.374955909 podStartE2EDuration="2.374955909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:51:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:51:05.372355566 +0000 UTC m=+1020.513368986" watchObservedRunningTime="2026-11-04 02:51:05.374955909 +0000 UTC m=+1020.515969317"
Nov 04 02:51:06.025113 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:06.025056    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:06.026932 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:06.026909    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 04 02:51:06.027046 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:06.026954    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 04 02:51:06.603999 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:51:06.603964    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:51:06.796013 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:06.795975    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:07.025179 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:07.025148    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 04 02:51:07.025354 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:07.025202    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 04 02:51:08.025423 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:08.025393    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 04 02:51:08.025790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:08.025448    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 04 02:51:09.025127 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:09.025096    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 04 02:51:09.025306 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:09.025146    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 04 02:51:10.025522 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:10.025488    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 04 02:51:10.025910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:10.025538    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 04 02:51:11.219366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:11.219326    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:11.221340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:11.221313    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:11.222292 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:11.222271    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 02:51:11.685940 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:11.685904    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:12.280274 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:12.280242    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:16.619475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:16.619439    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:17.637504 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.637472    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 02:51:17.641937 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.641918    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.710531 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.710501    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a0bcbc\"/\"openshift-service-ca.crt\""
Nov 04 02:51:17.710717 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.710566    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a0bcbc\"/\"default-dockercfg-fr85m\""
Nov 04 02:51:17.724122 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.724093    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:17.740501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740480    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttp7f\" (UniqueName: \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.740572 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740555    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.740661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740626    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.740784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740666    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.740784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740765    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.740871 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.740838    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.747020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.746996    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a0bcbc\"/\"distinct-2-w2-a0bcbc-kserve-self-signed-certs\""
Nov 04 02:51:17.747020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.747003    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a0bcbc\"/\"kube-root-ca.crt\""
Nov 04 02:51:17.749130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.749113    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 02:51:17.842035 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.841997    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttp7f\" (UniqueName: \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842058    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842172    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842201    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842244    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842316    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842550    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842687 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842665    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.842892 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.842867    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.844819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.844791    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.845174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.845153    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.902730 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.902674    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttp7f\" (UniqueName: \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\") pod \"distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") " pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:17.953857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:17.953828    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:18.416016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:18.415986    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 02:51:18.418858 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:51:18.418833    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0405bd41_0c36_43a5_9aef_41f06a6bc22b.slice/crio-eb74f3dce463113d6c1d357587fe79b75664069e7ea24969def97487af2caa79 WatchSource:0}: Error finding container eb74f3dce463113d6c1d357587fe79b75664069e7ea24969def97487af2caa79: Status 404 returned error can't find the container with id eb74f3dce463113d6c1d357587fe79b75664069e7ea24969def97487af2caa79
Nov 04 02:51:18.689942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:18.689851    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:19.356676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:19.356643    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" event={"ID":"0405bd41-0c36-43a5-9aef-41f06a6bc22b","Type":"ContainerStarted","Data":"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"}
Nov 04 02:51:19.356884 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:19.356681    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" event={"ID":"0405bd41-0c36-43a5-9aef-41f06a6bc22b","Type":"ContainerStarted","Data":"eb74f3dce463113d6c1d357587fe79b75664069e7ea24969def97487af2caa79"}
Nov 04 02:51:19.476178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:19.476117    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podStartSLOduration=2.476102049 podStartE2EDuration="2.476102049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:51:19.474329131 +0000 UTC m=+1034.615342538" watchObservedRunningTime="2026-11-04 02:51:19.476102049 +0000 UTC m=+1034.617115457"
Nov 04 02:51:19.571003 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:19.570968    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:19.954311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:19.954279    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:21.480396 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:21.480364    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:26.383695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:26.383662    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:27.954499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:27.954465    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:27.967254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:27.967228    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:28.416365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:28.416335    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:51:28.652123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:28.652054    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:31.284725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:31.284677    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:31.431511 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:31.431464    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:33.406984 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:33.406940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:34.271617 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.271583    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 02:51:34.275715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.275699    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.328460 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.328439    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ebb7fd-openshift-default-dockercfg-k9smx\""
Nov 04 02:51:34.380593 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.380565    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 02:51:34.434646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434613    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434669    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n49rp\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434702    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434843    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434884    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.434977    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.435013    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.435032    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.435090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.435083    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536241    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536289    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49rp\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536334    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536372    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536492    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536599    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536637    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536715 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536660    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536715    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536784    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.536969 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536879    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.537029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.536965    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.537196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.537152    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.537694 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.537668    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.539022 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.539000    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.539200 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.539183    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.596931 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.596901    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.597023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.597015    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49rp\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:34.890597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:34.890514    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:35.324522 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.324494    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 02:51:35.362034 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:51:35.362011    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdde55a0_318f_45be_ab19_41f2ea1e8f94.slice/crio-8628b1714ff73a9880c8977748e05612627ab09f3d850189a0506762a5807b82 WatchSource:0}: Error finding container 8628b1714ff73a9880c8977748e05612627ab09f3d850189a0506762a5807b82: Status 404 returned error can't find the container with id 8628b1714ff73a9880c8977748e05612627ab09f3d850189a0506762a5807b82
Nov 04 02:51:35.364438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.364407    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:35.364540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.364473    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:35.364540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.364527    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:35.443851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.443815    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" event={"ID":"fdde55a0-318f-45be-ab19-41f2ea1e8f94","Type":"ContainerStarted","Data":"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"}
Nov 04 02:51:35.444271 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.443877    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" event={"ID":"fdde55a0-318f-45be-ab19-41f2ea1e8f94","Type":"ContainerStarted","Data":"8628b1714ff73a9880c8977748e05612627ab09f3d850189a0506762a5807b82"}
Nov 04 02:51:35.564942 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.564884    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podStartSLOduration=1.564870704 podStartE2EDuration="1.564870704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:51:35.563196365 +0000 UTC m=+1050.704209773" watchObservedRunningTime="2026-11-04 02:51:35.564870704 +0000 UTC m=+1050.705884110"
Nov 04 02:51:35.891429 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:35.891394    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:36.200446 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:36.200363    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:36.609155 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:51:36.609119    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:51:36.892658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:36.892583    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:36.892658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:36.892633    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:37.891437 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:37.891405    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:37.891817 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:37.891467    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:38.891469 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:38.891437    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:38.891944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:38.891494    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:39.891519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:39.891479    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:39.891910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:39.891549    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:40.745536 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:40.745491    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:40.905229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:40.905186    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:40.905814 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:40.905781    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:41.117286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:41.117244    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:41.900661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:41.900626    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:41.900989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:41.900959    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:42.891824 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:42.891775    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 02:51:42.892373 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:42.891839    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 02:51:43.896215 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:43.896180    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:43.896709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:43.896678    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:43.897408 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:43.897384    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 02:51:46.686708 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:46.686677    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:48.503236 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.503204    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 02:51:48.507082 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.507054    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.675725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.675693    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.675923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.675756    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565m8\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.675923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.675871    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.675923 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.675909    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.676105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.676055    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.676156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.676114    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.676156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.676146    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.676259 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.676211    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.676345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.676330    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.707994 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.707965    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 02:51:48.777008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.776930    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777040    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777062    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777152    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565m8\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777185    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777218    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777295    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777352    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777439    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777772 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777723    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777772 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777750    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777805    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777841    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.777928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.777915    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.779690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.779668    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.780091 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.780054    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.838789 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.838756    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-565m8\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:48.838953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:48.838804    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:49.120628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.120547    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:49.550823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.550798    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 02:51:49.552765 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:51:49.552732    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb95735c_4dea_4c41_adbf_2fddd9288f2b.slice/crio-322d790a733e61273ce5c269c030dae66dc36c659877efca43e244549aed9997 WatchSource:0}: Error finding container 322d790a733e61273ce5c269c030dae66dc36c659877efca43e244549aed9997: Status 404 returned error can't find the container with id 322d790a733e61273ce5c269c030dae66dc36c659877efca43e244549aed9997
Nov 04 02:51:49.555533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.555504    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:49.555621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.555568    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:49.555621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.555600    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:51:49.574060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:49.574033    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" event={"ID":"bb95735c-4dea-4c41-adbf-2fddd9288f2b","Type":"ContainerStarted","Data":"322d790a733e61273ce5c269c030dae66dc36c659877efca43e244549aed9997"}
Nov 04 02:51:50.581643 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:50.581602    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" event={"ID":"bb95735c-4dea-4c41-adbf-2fddd9288f2b","Type":"ContainerStarted","Data":"2ca5ba1bee21d377ed6f5aa2131608f41cd48b724144dd0301ab2f7f2a54620c"}
Nov 04 02:51:50.823736 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:50.823684    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podStartSLOduration=2.82367144 podStartE2EDuration="2.82367144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:51:50.773746906 +0000 UTC m=+1065.914760314" watchObservedRunningTime="2026-11-04 02:51:50.82367144 +0000 UTC m=+1065.964684846"
Nov 04 02:51:51.121018 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:51.120986    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:51.122721 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:51.122696    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:51.122859 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:51.122759    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:51.630045 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:51.630013    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:52.121235 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:52.121191    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:52.121405 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:52.121253    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:53.121472 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:53.121439    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:53.121845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:53.121492    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:54.120891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:54.120859    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:54.121092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:54.120907    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:55.121453 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:55.121420    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:55.121861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:55.121470    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:56.121676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:56.121644    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 02:51:56.122164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:56.121694    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 02:51:56.687353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:56.687321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:51:57.127484 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:57.127455    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:57.128099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:57.127804    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:51:57.128725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:51:57.128661    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 02:52:01.586403 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:01.586368    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:02.922690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:02.922660    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 02:52:02.926890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:02.926869    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.053607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.053577    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 02:52:03.054265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054236    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054278    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054303    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbzlq\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054343    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054402 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054389    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054546    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054576    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054703 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054636    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.054703 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.054670    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.155914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.155836    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.155914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.155895    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.155957    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.155992    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156050    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156162    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156195    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbzlq\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156240    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156288    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156687 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156620    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.156957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.156933    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.157250 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.157223    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.157364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.157335    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.157457 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.157439    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.158830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.158805    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.159425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.159399    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.228279 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.228209    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbzlq\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.233536 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.233511    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 02:52:03.235259 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.235229    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\") pod \"e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") " pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.238796 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.238762    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.242520 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.242500    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:03.328123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.323279    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 02:52:03.359928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.359843    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360052    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360158 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360107    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360211 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360192    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360235    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t98qp\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360445 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360416    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360460    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360493    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.360533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.360520    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461417 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461385    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461428    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t98qp\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461502    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461520    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461539    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461622    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461806 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461667    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461929 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461805    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461929 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461832    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461929 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461834    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.461929 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.461871    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.462176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.462147    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.462237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.462194    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.462457 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.462434    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.464466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.464446    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.464684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.464668    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.522491 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.522468    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.522746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.522727    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t98qp\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.580658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.580631    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:03.694184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.694133    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:03.694364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.694279    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:03.694438 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.694362    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:03.697409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.697379    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 02:52:03.962100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.961993    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 02:52:03.965357 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:52:03.965330    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bf2f90_fed5_4c52_9089_f17620e4fa07.slice/crio-770ffd3e2d9b34089b1d9498af286b3fad2cc4a1cf2a257521d0f0ca20533460 WatchSource:0}: Error finding container 770ffd3e2d9b34089b1d9498af286b3fad2cc4a1cf2a257521d0f0ca20533460: Status 404 returned error can't find the container with id 770ffd3e2d9b34089b1d9498af286b3fad2cc4a1cf2a257521d0f0ca20533460
Nov 04 02:52:03.968103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.968052    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:03.968174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.968136    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:03.968174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:03.968167    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:04.681895 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.681847    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" event={"ID":"c3bf2f90-fed5-4c52-9089-f17620e4fa07","Type":"ContainerStarted","Data":"72df9bc35e5406252b36863d219029639a591cf107be404608fb22c821bbbccf"}
Nov 04 02:52:04.682130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.681920    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" event={"ID":"c3bf2f90-fed5-4c52-9089-f17620e4fa07","Type":"ContainerStarted","Data":"770ffd3e2d9b34089b1d9498af286b3fad2cc4a1cf2a257521d0f0ca20533460"}
Nov 04 02:52:04.683681 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.683656    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" event={"ID":"29f0cb07-c8c9-4b50-a467-c328951ace6c","Type":"ContainerStarted","Data":"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"}
Nov 04 02:52:04.683783 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.683699    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" event={"ID":"29f0cb07-c8c9-4b50-a467-c328951ace6c","Type":"ContainerStarted","Data":"12b89d2de9b0204ac422bcd5aacb29a6798f5627ec2d306b33fa2b9554398ca8"}
Nov 04 02:52:04.806882 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.806802    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podStartSLOduration=1.806786579 podStartE2EDuration="1.806786579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:52:04.803280722 +0000 UTC m=+1079.944294129" watchObservedRunningTime="2026-11-04 02:52:04.806786579 +0000 UTC m=+1079.947799985"
Nov 04 02:52:04.931889 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:04.931834    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podStartSLOduration=2.931817284 podStartE2EDuration="2.931817284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:52:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:52:04.929590877 +0000 UTC m=+1080.070604285" watchObservedRunningTime="2026-11-04 02:52:04.931817284 +0000 UTC m=+1080.072830692"
Nov 04 02:52:05.242732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.242703    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:05.244280 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.244248    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:05.244398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.244318    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:05.590095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.589425    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:05.595096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.592929    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:05.595096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:05.592984    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:06.244963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:06.243517    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:06.244963 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:06.243566    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:06.586216 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:06.586180    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:06.586844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:06.586455    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:06.616494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:06.616223    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:06.624017 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:52:06.623987    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:52:07.243717 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:07.243678    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:07.243912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:07.243738    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:07.582120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:07.581526    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:07.582120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:07.581587    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:08.243531 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:08.243494    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:08.243711 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:08.243541    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:08.582547 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:08.582507    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:08.583187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:08.583154    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:09.242940 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:09.242908    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:09.243222 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:09.243196    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:09.588097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:09.587965    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:09.588097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:09.588030    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:10.243126 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:10.243052    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 02:52:10.243319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:10.243158    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 02:52:10.581611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:10.581562    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:52:10.581834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:10.581640    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:52:11.311104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.311055    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:11.311864 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.311822    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:11.312844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.312823    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 02:52:11.809697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.809659    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:11.810478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.810457    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:11.811686 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.811662    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:11.812729 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:11.812695    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 02:52:16.788760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:16.788715    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:17.767988 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.767948    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 02:52:17.771975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.771951    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.825903 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.825862    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826021    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8wl\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826100    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826166    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826314    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826357    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826412    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826497    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.826567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.826543    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.878936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.878910    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 02:52:17.928249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928217    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928414 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928256    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928414 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928277    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928414 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928318    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928414 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928365    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928428    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928501    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8wl\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928532    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928591    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928737    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.928810 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928802    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.929191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.928950    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.929191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.929028    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.929428 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.929410    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.931341 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.931317    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.931459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.931444    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.990057 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.990030    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:17.990328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:17.990301    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8wl\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:18.086892 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.086717    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:18.520893 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.520870    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 02:52:18.523246 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:52:18.523218    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2c92fc_84e0_4959_9eb2_59fcc6572539.slice/crio-5c5bbc1effe609b5cfb30fcfe8f1fd50f5928d2eb998f2394897faf25cae27aa WatchSource:0}: Error finding container 5c5bbc1effe609b5cfb30fcfe8f1fd50f5928d2eb998f2394897faf25cae27aa: Status 404 returned error can't find the container with id 5c5bbc1effe609b5cfb30fcfe8f1fd50f5928d2eb998f2394897faf25cae27aa
Nov 04 02:52:18.525807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.525774    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:18.525909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.525840    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:18.525909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.525869    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:52:18.884554 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.884467    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" event={"ID":"8b2c92fc-84e0-4959-9eb2-59fcc6572539","Type":"ContainerStarted","Data":"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"}
Nov 04 02:52:18.884554 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:18.884532    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" event={"ID":"8b2c92fc-84e0-4959-9eb2-59fcc6572539","Type":"ContainerStarted","Data":"5c5bbc1effe609b5cfb30fcfe8f1fd50f5928d2eb998f2394897faf25cae27aa"}
Nov 04 02:52:19.013955 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:19.013871    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podStartSLOduration=2.013848517 podStartE2EDuration="2.013848517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:52:19.005392637 +0000 UTC m=+1094.146406045" watchObservedRunningTime="2026-11-04 02:52:19.013848517 +0000 UTC m=+1094.154861926"
Nov 04 02:52:19.087937 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:19.087902    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:19.089255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:19.089234    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:19.089314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:19.089294    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:20.087119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:20.087086    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:20.087530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:20.087133    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:21.087613 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:21.087583    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:21.088053 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:21.087629    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:21.839377 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:21.839338    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:22.087894 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:22.087864    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:22.088300 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:22.087914    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:23.087685 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:23.087653    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:23.087880 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:23.087710    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:24.087385 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:24.087353    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:24.087760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:24.087403    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:25.087615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:25.087578    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:52:25.088166 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:25.087627    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:52:26.092343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:26.092308    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:26.092760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:26.092740    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:26.093471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:26.093451    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 02:52:26.813096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:26.808642    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:30.382683 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:30.382641    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:31.641251 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:31.641191    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:31.808422 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:31.808390    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:33.777501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:33.777471    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:36.603126 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:52:36.603082    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:52:36.735147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:36.735116    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:41.725888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:41.725792    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:46.820051 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:46.820021    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:51.681782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:51.681727    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:52:56.590093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:52:56.590043    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:01.493255 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:01.493222    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:03.119144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.119113    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 02:53:03.122899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.122882    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.226404 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.226376    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 02:53:03.248381 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248350    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248426    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248482    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248542 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248503    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248670 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248602    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248670 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248627    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248768 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248728    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248813 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248780    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.248813 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.248803    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72x9t\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350185 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350153    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350227    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350259    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350286    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350364    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350401    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350483    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350519    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350555    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72x9t\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350608    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350704    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350750    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.350931 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350844    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.351014 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.350990    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.352830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.352801    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.353152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.353133    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.412247 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.412181    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.412355 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.412303    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72x9t\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\") pod \"e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") " pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.437936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.437910    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:03.866312 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.866276    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 02:53:03.869979 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:03.869953    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b50a92_262f_40ac_9eca_0f3e7805bff6.slice/crio-64597156dee583cb5d5928c2d88543394b2146bb2d3c80295332ac82130dbe3e WatchSource:0}: Error finding container 64597156dee583cb5d5928c2d88543394b2146bb2d3c80295332ac82130dbe3e: Status 404 returned error can't find the container with id 64597156dee583cb5d5928c2d88543394b2146bb2d3c80295332ac82130dbe3e
Nov 04 02:53:03.872409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.872379    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:03.872475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.872461    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:03.872509 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:03.872490    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:04.213305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.213199    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" event={"ID":"50b50a92-262f-40ac-9eca-0f3e7805bff6","Type":"ContainerStarted","Data":"aafd2acb02aeeaf82de8c979543fe1dc8760eeaf54131199259dc8d88c51b2a8"}
Nov 04 02:53:04.213305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.213285    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" event={"ID":"50b50a92-262f-40ac-9eca-0f3e7805bff6","Type":"ContainerStarted","Data":"64597156dee583cb5d5928c2d88543394b2146bb2d3c80295332ac82130dbe3e"}
Nov 04 02:53:04.333112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.333033    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podStartSLOduration=1.333019876 podStartE2EDuration="1.333019876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:04.331828123 +0000 UTC m=+1139.472841541" watchObservedRunningTime="2026-11-04 02:53:04.333019876 +0000 UTC m=+1139.474033283"
Nov 04 02:53:04.438213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.438183    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:04.439829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.439807    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:04.439937 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:04.439866    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:05.438451 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:05.438420    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:05.438920 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:05.438476    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:06.405263 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:06.405231    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:06.442773 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:06.442738    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:06.449564 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:06.444834    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:06.617495 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:53:06.617457    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:53:06.983789 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:06.983743    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:07.443643 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:07.443602    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:07.444230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:07.444195    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:08.438289 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:08.438258    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:08.438606 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:08.438318    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:09.438394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:09.438354    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:09.438924 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:09.438412    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:10.438242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:10.438208    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:53:10.438420 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:10.438257    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:53:11.385061 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:11.385020    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:12.305463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.305427    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 02:53:12.309856 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.309833    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.390920 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.390885    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:12.416262 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.416238    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-e516ff\"/\"distinct-2-w1-e516ff-kserve-self-signed-certs\""
Nov 04 02:53:12.418917 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.418890    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 02:53:12.429712 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.429660    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.429712 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.429708    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.429945 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.429919    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56c6q\" (UniqueName: \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.430089 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.430045    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.430234 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.430216    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.430286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.430248    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.509925 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.509879    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:53:12.510229 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.510199    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:12.545205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.545167    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.545499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.545473    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.545684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.545669    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56c6q\" (UniqueName: \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.550262 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.550229    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.557517 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.550551    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.557723 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.557701    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.558039 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.551626    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.558191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.551409    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.565581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.558558    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.569087 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.567364    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.596690 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.596655    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.612962 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.612931    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56c6q\" (UniqueName: \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\") pod \"distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") " pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:12.622988 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:12.622962    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:13.038382 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.038351    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:13.038815 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.038791    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:13.039757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.039733    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 02:53:13.209888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.209798    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 02:53:13.247157 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:13.247132    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc095802_ca07_4d29_8d08_0c82006f2541.slice/crio-dcea90d07194debbe7100018e580c4b11a8c419acc726eb2462796ec271a73f9 WatchSource:0}: Error finding container dcea90d07194debbe7100018e580c4b11a8c419acc726eb2462796ec271a73f9: Status 404 returned error can't find the container with id dcea90d07194debbe7100018e580c4b11a8c419acc726eb2462796ec271a73f9
Nov 04 02:53:13.342837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.342794    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" event={"ID":"fc095802-ca07-4d29-8d08-0c82006f2541","Type":"ContainerStarted","Data":"8fab2b144f349f3eba7836a46921bda4d7dc4f3bcfdfc8bd6e5bc3af5ec1725a"}
Nov 04 02:53:13.343288 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.342887    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" event={"ID":"fc095802-ca07-4d29-8d08-0c82006f2541","Type":"ContainerStarted","Data":"dcea90d07194debbe7100018e580c4b11a8c419acc726eb2462796ec271a73f9"}
Nov 04 02:53:13.463124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.462949    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podStartSLOduration=1.4629314 podStartE2EDuration="1.4629314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:13.462607639 +0000 UTC m=+1148.603621047" watchObservedRunningTime="2026-11-04 02:53:13.4629314 +0000 UTC m=+1148.603944808"
Nov 04 02:53:13.624871 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:13.624679    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:16.289136 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:16.289101    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:20.113996 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.113961    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 02:53:20.117954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.117937    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265030 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.264999    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265030 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265032    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265050    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265232    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265256    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265293    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8qjz\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265477 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265453    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265505    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.265578 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.265567    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.328766 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.328730    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 02:53:20.367343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367248    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367309    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367363    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8qjz\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367471    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367525    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367575    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367635    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367663    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.367724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367691    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.368243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.367983    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.368243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.368103    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.368243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.368189    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.368243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.368219    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.368243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.368223    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.370047 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.370025    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.370193 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.370170    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.428627 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.428602    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.428765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.428692    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8qjz\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.432599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.432574    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:20.863733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.863703    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 02:53:20.865427 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:20.865399    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6709b764_e7ba_48a6_b730_f46cff868bbd.slice/crio-93ff34759ab6df9152ae3a9495a132ed6b7b174ed9477e55782e091345aff791 WatchSource:0}: Error finding container 93ff34759ab6df9152ae3a9495a132ed6b7b174ed9477e55782e091345aff791: Status 404 returned error can't find the container with id 93ff34759ab6df9152ae3a9495a132ed6b7b174ed9477e55782e091345aff791
Nov 04 02:53:20.868111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.868080    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:20.868196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.868146    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:20.868196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:20.868179    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:21.202485 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.202410    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:21.423303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.423263    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" event={"ID":"6709b764-e7ba-48a6-b730-f46cff868bbd","Type":"ContainerStarted","Data":"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"}
Nov 04 02:53:21.423475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.423333    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" event={"ID":"6709b764-e7ba-48a6-b730-f46cff868bbd","Type":"ContainerStarted","Data":"93ff34759ab6df9152ae3a9495a132ed6b7b174ed9477e55782e091345aff791"}
Nov 04 02:53:21.433769 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.433748    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:21.434945 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.434920    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:21.435043 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.434962    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:21.544100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:21.544026    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podStartSLOduration=1.5440063990000001 podStartE2EDuration="1.544006399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:21.542110552 +0000 UTC m=+1156.683123961" watchObservedRunningTime="2026-11-04 02:53:21.544006399 +0000 UTC m=+1156.685019809"
Nov 04 02:53:22.433905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:22.433865    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:22.434311 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:22.433920    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:22.623180 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:22.623149    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:22.635704 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:22.635677    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:23.228254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:23.228223    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:23.434117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:23.433981    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:23.434117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:23.434055    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:23.466857 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:23.466738    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:24.433529 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:24.433497    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:24.433709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:24.433551    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:25.433499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:25.433461    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:25.434209 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:25.434158    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:26.447098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:26.446982    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:26.447098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:26.447046    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:26.674097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:26.673327    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:27.099940 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.099887    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.165088 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.153551    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.166876 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.166831    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.178243 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.178209    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:53:27.178459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.178430    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.408301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.408197    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.419099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.415166    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:53:27.419099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.415239    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.437526 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.437492    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:27.437774 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.437746    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:27.446217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.446159    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.526161 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.526119    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:27.675272 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:27.675166    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.166110 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.119098    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.174100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.166771    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:53:28.370464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.364678    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:53:28.370464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.364736    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.370464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.176852    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.397124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.174353    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.397812 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.397789    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:53:28.398028 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.398013    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:53:28.509365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.509320    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.596581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.596536    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.597503 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.597482    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:28.606254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.606228    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:53:28.638095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.631335    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.642377 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.642338    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:53:28.684980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.684932    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:53:28.719571 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.719532    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:53:28.742610 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.727460    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:28.742610 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.727508    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:28.749859 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.749818    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:53:28.750413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.750393    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:53:28.773360 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.762661    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:53:28.773360 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.768465    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:53:28.773360 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.768512    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:53:28.778467 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.778389    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:53:28.794788 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:28.784149    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:53:29.052968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:29.052792    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:29.433048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:29.432961    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 04 02:53:29.433048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:29.433022    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 04 02:53:30.611274 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:30.611235    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:30.611769 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:30.611684    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:30.613433 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:30.613407    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 02:53:31.994302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:31.994269    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:32.158584 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:32.156150    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:34.120031 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:34.119998    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:36.602715 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:53:36.602677    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:53:36.998412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:36.998336    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:41.822585 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:41.822542    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:44.944351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:44.944316    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 02:53:44.949339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:44.949322    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.007596 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.007563    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 02:53:45.045506 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.045479    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:45.053051 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.053030    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b7476\"/\"distinct-2-w5-3b7476-kserve-self-signed-certs\""
Nov 04 02:53:45.084754 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.084719    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.084874 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.084805    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.084874 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.084844    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.085144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.085121    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.085272 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.085248    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s8tc\" (UniqueName: \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.085359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.085345    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187198    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187319    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187407    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187477    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187578    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.188098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.187659    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s8tc\" (UniqueName: \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.192099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.188834    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.195772 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.195587    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.195909 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.195886    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.196295 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.196269    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.197844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.197819    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.251358 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.251325    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s8tc\" (UniqueName: \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\") pod \"distinct-2-w5-3b7476-kserve-799c549f96-qhvzt\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") " pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.262205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.262177    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:45.864873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.864851    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 02:53:45.867128 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:45.867063    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod769a6f5a_1811_4106_a8a7_b1636a39218b.slice/crio-133937624a0c434abecbb594da9c3d340da5406c25713a76a425bc513021abcc WatchSource:0}: Error finding container 133937624a0c434abecbb594da9c3d340da5406c25713a76a425bc513021abcc: Status 404 returned error can't find the container with id 133937624a0c434abecbb594da9c3d340da5406c25713a76a425bc513021abcc
Nov 04 02:53:45.990089 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.990043    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" event={"ID":"769a6f5a-1811-4106-a8a7-b1636a39218b","Type":"ContainerStarted","Data":"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"}
Nov 04 02:53:45.990474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:45.990102    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" event={"ID":"769a6f5a-1811-4106-a8a7-b1636a39218b","Type":"ContainerStarted","Data":"133937624a0c434abecbb594da9c3d340da5406c25713a76a425bc513021abcc"}
Nov 04 02:53:46.107115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:46.107040    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podStartSLOduration=2.107028003 podStartE2EDuration="2.107028003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:46.106574016 +0000 UTC m=+1181.247587424" watchObservedRunningTime="2026-11-04 02:53:46.107028003 +0000 UTC m=+1181.248041409"
Nov 04 02:53:46.264264 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:46.264190    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:46.768084 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:46.768038    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:49.423667 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.423636    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 02:53:49.427426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.427406    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486531    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486566    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486795 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486678    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486795 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486735    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486809    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf4qx\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.486914 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486850    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.487023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.486981    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.487135 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.487027    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.487135 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.487113    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.587928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.587895    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.587967    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.587997    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588098    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588124    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588178    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf4qx\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588245    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588343    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588392    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588519 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588489    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588721 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588701    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588889    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.588935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.588919    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.589153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.589129    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.590957 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.590934    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.591141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.591120    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.635906 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.635875    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 02:53:49.647115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.647041    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 02:53:49.652463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.652432    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.652887 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.652863    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.654953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.654925    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf4qx\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\") pod \"e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") " pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.689445 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689414    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.689592 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689460    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.689699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689679    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.689799 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689784    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.689897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689863    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.690018 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.689974    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl5jm\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.690018 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.690002    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.690169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.690029    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.690169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.690138    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.711425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.711401    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 02:53:49.742355 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.742291    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.792866    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.792943    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793005    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793036    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793162    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793237    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793299    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793396    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl5jm\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.795778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.793457    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.796499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.796467    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.796802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.796763    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.797047 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.797023    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.797262 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.797239    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.797436 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.797418    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.797562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.797535    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.802728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.798061    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.857504 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.857476    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.857689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.857668    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl5jm\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:49.994822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:49.994746    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:50.188640 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.188612    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 02:53:50.190481 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:50.190448    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d72228_116e_4804_a8c6_09d84e814427.slice/crio-7ce54e9de8fad9a374fcaf1671ff8c6f315e83fd99120f077e40174ad07ef13a WatchSource:0}: Error finding container 7ce54e9de8fad9a374fcaf1671ff8c6f315e83fd99120f077e40174ad07ef13a: Status 404 returned error can't find the container with id 7ce54e9de8fad9a374fcaf1671ff8c6f315e83fd99120f077e40174ad07ef13a
Nov 04 02:53:50.193348 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.193313    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:50.193447 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.193377    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:50.193447 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.193410    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:50.377717 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.377683    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 02:53:50.379267 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:53:50.379243    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa9c31e_9c6c_4c24_97a9_a395cb5a83c3.slice/crio-b02a55bbc6d79094ec66d6516c4dd44410a2969a5d6776246c838f913796ae23 WatchSource:0}: Error finding container b02a55bbc6d79094ec66d6516c4dd44410a2969a5d6776246c838f913796ae23: Status 404 returned error can't find the container with id b02a55bbc6d79094ec66d6516c4dd44410a2969a5d6776246c838f913796ae23
Nov 04 02:53:50.381865 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.381820    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:50.381955 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.381904    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:50.381955 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:50.381932    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:53:51.025928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.025885    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" event={"ID":"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3","Type":"ContainerStarted","Data":"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"}
Nov 04 02:53:51.026406 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.025949    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" event={"ID":"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3","Type":"ContainerStarted","Data":"b02a55bbc6d79094ec66d6516c4dd44410a2969a5d6776246c838f913796ae23"}
Nov 04 02:53:51.027620 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.027597    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" event={"ID":"12d72228-116e-4804-a8c6-09d84e814427","Type":"ContainerStarted","Data":"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"}
Nov 04 02:53:51.027695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.027642    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" event={"ID":"12d72228-116e-4804-a8c6-09d84e814427","Type":"ContainerStarted","Data":"7ce54e9de8fad9a374fcaf1671ff8c6f315e83fd99120f077e40174ad07ef13a"}
Nov 04 02:53:51.154971 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.154906    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podStartSLOduration=2.154887755 podStartE2EDuration="2.154887755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:51.150546331 +0000 UTC m=+1186.291559739" watchObservedRunningTime="2026-11-04 02:53:51.154887755 +0000 UTC m=+1186.295901163"
Nov 04 02:53:51.284165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.283679    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podStartSLOduration=2.283661954 podStartE2EDuration="2.283661954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:53:51.281770592 +0000 UTC m=+1186.422784000" watchObservedRunningTime="2026-11-04 02:53:51.283661954 +0000 UTC m=+1186.424675361"
Nov 04 02:53:51.700834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.700728    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:51.769095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.765522    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:51.769095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.765660    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:51.769095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.765704    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:52.003156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.998289    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:52.003156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.998447    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:52.003156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:51.998490    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:52.742741 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:52.742709    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:52.743245 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:52.742757    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:53.000179 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:52.996349    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:53.000179 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:52.996403    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:53.743552 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:53.743520    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:53.744012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:53.743576    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:53.996636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:53.996540    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:53.996893 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:53.996864    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:54.742790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:54.742748    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:54.743001 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:54.742819    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:54.921500 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:54.921467    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:55.001191 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.001155    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:55.001380 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.001216    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:55.091353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.091321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:55.265633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.265550    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:55.310013 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.309985    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:55.743302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.743220    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:55.743302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.743281    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:55.997527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.997442    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:55.998131 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:55.998103    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:56.119893 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.119866    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:53:56.714302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.714256    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:53:56.743582 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.743541    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:56.743809 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.743610    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:56.995874 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.995794    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:53:56.995874 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:56.995853    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:53:57.743463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:57.743426    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:53:57.744007 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:57.743493    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:53:58.211230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.211196    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:58.212908 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.212112    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:58.213237 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.213215    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 02:53:58.747302 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.747274    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:58.747747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.747734    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:53:58.748356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:53:58.748335    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 02:54:02.197097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.196574    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:02.691301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.691237    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.728390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.728346    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.773003 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.772165    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.773319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.773287    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.774417 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.773728    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:54:02.781807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.781138    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.785162 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.785136    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:02.866737 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:02.866698    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:54:03.100719 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.100679    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.152153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.151977    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.457340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.457234    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.705193 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.695478    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.705193 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.698632    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Nov 04 02:54:03.705193 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.700765    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:54:03.769153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.729354    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.800149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.786498    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.819102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.800743    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:54:03.819419 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.819393    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:54:03.862098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.786595    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 04 02:54:03.862684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.786531    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 02:54:03.878264 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.878234    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:54:03.878565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.822329    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:54:03.878750 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.878734    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:54:03.886095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.882154    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:54:03.916107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.879991    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:03.925096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.786561    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:54:03.944096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.925617    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:54:03.944392 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:03.944371    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:54:04.151344 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.151240    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.189187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.164409    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:54:04.669055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.667537    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:54:04.669055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.668101    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:54:04.705092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.699941    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.799099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.796033    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.814097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.811459    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.874108 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.855632    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 02:54:04.877361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.864785    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.889209 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.889163    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.913811 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.912191    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:04.924172 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.923280    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:54:04.950338 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:04.950295    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:05.203020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.172139    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:05.203020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.172296    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 02:54:05.203020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.179964    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:05.203020 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.181738    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237291    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237408    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237471    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237607 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237525    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237616    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237656    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237683    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.237786 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237754    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.238048 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.237834    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npzlb\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.429097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.415906    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.495101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.494987    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.495979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.495948    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496041    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496218 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496151    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npzlb\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496218 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496188    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496294    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496387 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496345    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.496387 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496385    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.497539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496722    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.497539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.417749    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.497539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.496919    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.502973 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.502587    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.512341 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.512309    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.512864 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.512838    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.513908 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.513663    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.564740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.564515    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:54:05.564740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.564563    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:54:05.564740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.564595    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:54:05.564740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.564627    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:54:05.564740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.564655    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:54:05.569570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.569540    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:54:05.570440 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.570414    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:54:05.636610 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.636576    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.657997 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.657961    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npzlb\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\") pod \"e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") " pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.675386 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.675349    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:54:05.683935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.675342    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:54:05.688055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.688023    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:05.717111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.710588    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:54:05.742116 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.742061    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:54:05.798193 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.798167    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:54:05.799202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:05.799185    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:54:06.187277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.187246    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 02:54:06.188050 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:54:06.188011    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0472db5_6406_489a_bfdb_47586179bb33.slice/crio-caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9 WatchSource:0}: Error finding container caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9: Status 404 returned error can't find the container with id caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9
Nov 04 02:54:06.191968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.191931    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:06.192096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.192009    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:06.192096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.192087    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:06.588345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.586742    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" event={"ID":"e0472db5-6406-489a-bfdb-47586179bb33","Type":"ContainerStarted","Data":"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"}
Nov 04 02:54:06.588345 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.586809    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" event={"ID":"e0472db5-6406-489a-bfdb-47586179bb33","Type":"ContainerStarted","Data":"caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9"}
Nov 04 02:54:06.622359 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:54:06.622324    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:54:06.696329 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:06.696185    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:07.182060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:07.181893    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podStartSLOduration=3.181864064 podStartE2EDuration="3.181864064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:54:07.18028872 +0000 UTC m=+1202.321302196" watchObservedRunningTime="2026-11-04 02:54:07.181864064 +0000 UTC m=+1202.322877468"
Nov 04 02:54:07.697488 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:07.697457    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:07.697855 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:07.697512    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:08.696511 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:08.696478    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:08.696680 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:08.696528    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:09.299395 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:09.299359    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:09.695933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:09.695814    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:09.695933 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:09.695878    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:10.696695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:10.696665    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:10.697059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:10.696713    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:11.696473 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:11.696437    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:11.696643 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:11.696491    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:12.696432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:12.696399    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 04 02:54:12.696834 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:12.696476    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 04 02:54:14.084209 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:14.084160    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:14.085001 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:14.084963    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:14.086870 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:14.086849    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 02:54:14.335609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:14.335536    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:19.313554 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:19.313523    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:24.347167 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:24.347136    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:29.184151 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:29.184113    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:32.402983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:32.402945    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:34.082467 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.082406    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:34.393726 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.393646    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 02:54:34.398901 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.398879    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.476636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.476601    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:34.492934 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.492888    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493036    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbm2\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493168 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493147    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493259 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493232    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493327    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493452 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493420    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493497    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493571 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493549    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.493621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.493572    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.509716 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.509689    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 02:54:34.594475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594443    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594509    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594532    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594581    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594661    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbm2\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594785    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594938 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594819    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594938 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594861    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.594938 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594919    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.595142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.594981    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.595248 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.595225    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.595307 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.595249    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.595354 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.595331    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.595459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.595441    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.597130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.597109    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.597375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.597359    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.655399 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.655333    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.655563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.655540    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbm2\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\") pod \"e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") " pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:34.713615 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:34.713586    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:35.174300 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.172005    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 02:54:35.213007 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.212901    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:35.213007 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.212996    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:35.213249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.213040    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:35.436155 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.429546    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 02:54:35.475109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.472346    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.512093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.509376    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734337    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734418    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbw95\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734490    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734585    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734730    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.734956    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.735162    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.735218    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.750101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.735321    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.842093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.841667    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbw95\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.842093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.841751    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.842093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.841814    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.842093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.841890    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.842093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.841963    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.842127    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.842163    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.842253    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.842366    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.843502    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.843735    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.844397    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.855118    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.858743    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.871093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.859267    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.961106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.958058    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:35.998145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:35.983950    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbw95\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:36.012261 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.005465    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" event={"ID":"aabe9ac3-8823-4883-b0f3-fe9b19562187","Type":"ContainerStarted","Data":"4a649331bf7e830ac3b05fa7ed2817a9e48d26ad971bfe14330c2b77886bb8b7"}
Nov 04 02:54:36.012261 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.005727    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" event={"ID":"aabe9ac3-8823-4883-b0f3-fe9b19562187","Type":"ContainerStarted","Data":"2367218138b6cf8167f0155c260e84e619ec5176cd39510c91531cb9f2d2e72a"}
Nov 04 02:54:36.014662 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.014624    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:36.043099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.040694    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:36.403107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.403045    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:36.719944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.714818    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:36.724096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.720186    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:36.724096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:36.720239    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:37.279098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.278273    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:37.283097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.279887    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:37.330989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.312735    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 04 02:54:37.340118 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:54:37.336230    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:54:37.433720 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.425829    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podStartSLOduration=3.425807732 podStartE2EDuration="3.425807732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:54:36.134154403 +0000 UTC m=+1231.275167813" watchObservedRunningTime="2026-11-04 02:54:37.425807732 +0000 UTC m=+1232.566821140"
Nov 04 02:54:37.528984 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.528930    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:37.538773 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.538685    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:37.539106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.539043    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:37.544780 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.532460    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:37.546249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.536128    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:54:37.550492 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.531996    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:54:37.568695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.568662    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 02:54:37.741784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.741552    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:37.742130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.742097    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:37.757115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.747160    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:37.808462 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.808363    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:54:37.835226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:37.835179    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.056976 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.055266    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:38.330599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.324170    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/health\": context deadline exceeded"
Nov 04 02:54:38.330599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.324270    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.330599 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.324597    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:38.339597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.332570    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:54:38.339597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.332638    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:54:38.339597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.336911    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.339597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.337253    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.567907    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.576637    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.583029    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.592174    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.593868    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:54:38.617119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.593976    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:54:38.720636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.720519    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:38.720636 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.720574    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:38.765825 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.755533    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:38.765825 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.756155    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:54:38.774980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.774945    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" event={"ID":"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b","Type":"ContainerStarted","Data":"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"}
Nov 04 02:54:38.775407 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.775384    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" event={"ID":"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b","Type":"ContainerStarted","Data":"86cb8c5c60b3c335d0588804f631d5bfdb2154f77a9d156161caafdbafe490c0"}
Nov 04 02:54:38.822249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.822197    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:38.829101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.822806    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": context deadline exceeded"
Nov 04 02:54:38.829417 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.829396    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:54:38.842505 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.842475    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:38.975918 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.939238    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:54:38.986109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:38.959179    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.074108 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.070160    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.143204 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.136409    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:39.143481 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.143449    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:39.349098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.348312    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:54:39.349098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.348768    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.374432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.370146    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.533099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.530606    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podStartSLOduration=4.530583878 podStartE2EDuration="4.530583878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:54:39.117058183 +0000 UTC m=+1234.258071588" watchObservedRunningTime="2026-11-04 02:54:39.530583878 +0000 UTC m=+1234.671597286"
Nov 04 02:54:39.602147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.590646    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.602147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.596468    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.628096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.608405    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.696318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.413060    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:39.696318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.634306    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:54:39.821324 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.821283    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:54:39.822375 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.822348    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:39.822560 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.822536    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:39.842000 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.835283    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:39.937170 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:39.934435    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.103097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.102746    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.128160 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.128125    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:40.128416 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.128390    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:40.207023 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.206936    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:40.351099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.348747    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.374307 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.373171    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.377684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.377654    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.621409 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.621363    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.622967 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.622937    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:54:40.629095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.624095    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:54:40.629871 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.629845    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:54:40.640125 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.640096    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 02:54:40.725060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.725029    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:40.725371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.725324    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:40.736720 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.736691    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:54:40.753569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.745965    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:54:40.753569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.747383    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:54:40.832129 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.831558    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.832129 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.831640    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:54:40.863565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.863536    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:54:40.936602 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.936498    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:40.964953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:40.964929    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:54:41.041503 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.041472    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:41.041738 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.041714    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:41.119097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.110094    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:54:41.176830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.176800    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:54:41.178289 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.178269    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:54:41.390927 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.390388    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:54:41.714528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.714441    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:41.729922 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:41.729884    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:42.041735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:42.041685    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:42.041934 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:42.041751    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:42.717678 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:42.717238    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:42.717678 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:42.717298    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:43.042031 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:43.042002    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:43.042233 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:43.042050    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:43.714684 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:43.714647    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:43.714869 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:43.714702    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:44.042165 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:44.042134    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:44.042752 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:44.042720    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:44.714367 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:44.714323    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:44.714531 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:44.714385    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:45.042527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:45.042493    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:45.042889 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:45.042542    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:45.063126 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:45.060539    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:45.714736 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:45.714692    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:54:45.714935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:45.714767    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:54:46.042206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:46.042170    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:54:46.042400 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:46.042246    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:54:46.721263 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:46.721229    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:46.721737 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:46.721582    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:46.722779 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:46.722757    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 02:54:47.164505 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:47.164473    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:47.165022 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:47.164989    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:47.166541 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:47.166516    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 02:54:48.594378 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:48.594340    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:49.979512 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:49.979479    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:50.853178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.853144    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 02:54:50.857335 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.857314    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.922133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.922036    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 02:54:50.990042 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990012    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990057    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990172    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990204    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990289    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990358    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krj2x\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990426 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990381    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990622 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990504    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:50.990622 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:50.990535    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091503 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091471    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091519    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091571    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091676 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091597    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091676    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091782 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091775    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krj2x\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.091873 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091811    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.092141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091948    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.092141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.091989    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.092390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.092349    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.092905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.092867    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.092996 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.092981    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.093059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.093020    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.093326 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.093056    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.094339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.094318    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.095177 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.095154    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.152835 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.152762    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.153277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.153254    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krj2x\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.171999 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.171975    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:51.602768 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.602739    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 02:54:51.604691 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:54:51.604656    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b13c4f3_cfc8_49b2_85ec_9b35670397a1.slice/crio-f0a771af75dd1c324f7225c7625245918f20f350630588d753eadeb96c27503c WatchSource:0}: Error finding container f0a771af75dd1c324f7225c7625245918f20f350630588d753eadeb96c27503c: Status 404 returned error can't find the container with id f0a771af75dd1c324f7225c7625245918f20f350630588d753eadeb96c27503c
Nov 04 02:54:51.607242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.607210    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:51.607339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.607289    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:51.607404 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:51.607346    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:54:52.398435 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:52.398397    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" event={"ID":"4b13c4f3-cfc8-49b2-85ec-9b35670397a1","Type":"ContainerStarted","Data":"8699ee630c550e6db01d98872656098656e6ef2fda5d16f72db5d1e637c35640"}
Nov 04 02:54:52.398906 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:52.398450    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" event={"ID":"4b13c4f3-cfc8-49b2-85ec-9b35670397a1","Type":"ContainerStarted","Data":"f0a771af75dd1c324f7225c7625245918f20f350630588d753eadeb96c27503c"}
Nov 04 02:54:52.623467 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:52.623418    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podStartSLOduration=2.623403111 podStartE2EDuration="2.623403111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:54:52.572726708 +0000 UTC m=+1247.713740118" watchObservedRunningTime="2026-11-04 02:54:52.623403111 +0000 UTC m=+1247.764416517"
Nov 04 02:54:53.173149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:53.172422    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:53.174760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:53.174721    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:53.174884 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:53.174780    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:54.173539 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:54.173504    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:54.173959 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:54.173606    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:55.107208 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:55.107176    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:54:55.173514 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:55.173475    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:55.173829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:55.173794    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:56.173609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:56.173569    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:56.173887 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:56.173647    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:57.173530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:57.173485    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:57.173751 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:57.173573    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:58.173059 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:58.173023    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:54:58.173496 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:58.173113    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 02:54:59.177398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:59.177367    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:59.177921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:59.177726    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:54:59.178619 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:54:59.178600    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 02:55:00.228616 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:00.228583    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:04.156278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.156235    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.387095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.382216    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.611213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.611167    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.624928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.624199    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:04.628196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.628161    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.670403 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.670025    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 02:55:04.756098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.755197    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.836099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.835178    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:04.941153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:04.940567    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:55:05.134417 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.134369    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:05.179099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.157154    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:05.179099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.157587    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:05.220154 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:55:05.208009    6019 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0472db5_6406_489a_bfdb_47586179bb33.slice/crio-caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9\": RecentStats: unable to find data in memory cache]"
Nov 04 02:55:05.529722 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.529668    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:05.761665 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.761612    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:05.790351 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.790258    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:55:05.791895 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.791870    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:05.792148 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.792135    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:55:05.818140 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.628009    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:55:05.848112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.848039    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:05.876377 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.876341    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:55:05.876700 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:05.876681    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:55:06.057277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.014133    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:06.322557 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.322460    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Nov 04 02:55:06.324709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.072569    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:55:06.379486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.379441    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:06.433588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.426472    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:55:06.434919 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.434883    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:55:06.444864 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.435989    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:06.460895 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.460861    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:06.518469 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.517099    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:06.535597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.518856    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:55:06.535597 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.519811    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:06.630332 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.630243    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:06.630900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.630878    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:55:06.768383 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.768342    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:06.798635 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.798586    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:06.902149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.897223    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:55:06.927679 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:06.927568    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:07.235611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.235523    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:07.334285 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.334237    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:07.431466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.431424    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:07.467099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.463005    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:07.559432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.544255    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:07.559432 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.544709    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:07.847160 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.821188    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:07.847160 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:55:07.822704    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:55:07.847805 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.847767    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:07.943314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.920484    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:55:07.953891 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:07.953840    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:55:08.451259 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.451211    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.486248 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.467180    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.495458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.467204    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:08.597278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.596490    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.597278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.596894    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:08.829094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.828226    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.829094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.828375    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.829094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.828838    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:55:08.851265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.851221    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:08.963213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.963165    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Nov 04 02:55:08.963534 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.963508    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:55:08.968019 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:08.967983    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:55:09.545268 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.545217    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:09.557095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.549802    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:55:09.611508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.611463    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:09.618553 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.611782    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:09.837609 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.837481    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:09.839478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.839440    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:09.840543 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.840518    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:09.859699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.859645    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:09.909744 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.909703    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:55:09.961101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.942244    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:09.963867 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:09.963805    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.018740 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.018696    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.576099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.549300    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.576099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.551950    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.593005 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.590271    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.619672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.618786    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:10.662770 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.662713    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.784549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.782539    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:10.786182 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.785156    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.840273 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.839115    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.869161 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.860431    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 04 02:55:10.885019 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.883449    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:10.973704 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:10.973258    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:11.041338 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.040271    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 02:55:11.091132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.091025    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:55:11.099132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.091400    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:55:11.219196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.211496    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:55:11.242724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.237530    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:11.242724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.237739    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:55:11.242724 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.241230    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:11.583641 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.560781    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:11.603152 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.600749    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:11.609142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.609103    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:11.618101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.616623    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:55:11.693847 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.692095    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:11.719415 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:11.719384    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:55:16.990242 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:16.990215    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:20.984746 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:20.984717    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 02:55:20.989677 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:20.989656    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.096130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.096094    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 02:55:21.130653 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.130475    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.130823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.130675    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp9vz\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.130823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.130743    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.130823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.130792    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.130976 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.130933    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.131045 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.131024    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.131122 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.131059    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.131122 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.131111    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.131230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.131207    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232354 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232325    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232365    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp9vz\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232387    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232415    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232528 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232510    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232540    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232559    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232590    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.232802 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232672    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.233016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232808    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.233016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232860    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.233016 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.232940    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.233185 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.233045    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.233332 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.233310    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.235003 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.234987    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.235157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.235142    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.296272 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.296237    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.298790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.298763    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp9vz\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.304678 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.304652    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:21.741590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.741562    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 02:55:21.743023 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:55:21.742999    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f68d9f7_8ed0_4687_b7a4_97745a8d0197.slice/crio-b8b9a09d7dd8aaf3c0777eb53d426e63adfbce0f396f6618bb55b654bbcea43c WatchSource:0}: Error finding container b8b9a09d7dd8aaf3c0777eb53d426e63adfbce0f396f6618bb55b654bbcea43c: Status 404 returned error can't find the container with id b8b9a09d7dd8aaf3c0777eb53d426e63adfbce0f396f6618bb55b654bbcea43c
Nov 04 02:55:21.745524 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.745489    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:55:21.745591 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.745569    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:55:21.745639 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.745610    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:55:21.876642 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:21.876609    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:22.233148 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.233108    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" event={"ID":"1f68d9f7-8ed0-4687-b7a4-97745a8d0197","Type":"ContainerStarted","Data":"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"}
Nov 04 02:55:22.233543 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.233174    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" event={"ID":"1f68d9f7-8ed0-4687-b7a4-97745a8d0197","Type":"ContainerStarted","Data":"b8b9a09d7dd8aaf3c0777eb53d426e63adfbce0f396f6618bb55b654bbcea43c"}
Nov 04 02:55:22.305552 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.305521    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:22.307198 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.307162    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:22.307340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.307223    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:22.354275 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:22.353324    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podStartSLOduration=2.353305694 podStartE2EDuration="2.353305694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:55:22.352575809 +0000 UTC m=+1277.493589216" watchObservedRunningTime="2026-11-04 02:55:22.353305694 +0000 UTC m=+1277.494319101"
Nov 04 02:55:23.305047 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:23.305009    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:23.305500 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:23.305066    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:24.305683 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:24.305643    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:24.306109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:24.305714    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:25.305483 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:25.305449    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:25.305664 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:25.305507    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:26.305875 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:26.305823    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:26.306328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:26.305892    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:26.903842 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:26.903808    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:27.305052 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:27.305019    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:27.305342 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:27.305090    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:28.276887 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:28.276856    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:28.305352 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:28.305316    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:55:28.305532 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:28.305367    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:55:28.423731 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:28.423692    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:29.059739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:29.059702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:29.311249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:29.311159    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:29.311733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:29.311654    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:29.312382 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:29.312363    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 02:55:31.894750 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:31.894716    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:32.703468 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:32.703415    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:32.872258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:32.872224    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:33.016384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:33.016354    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:33.161344 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:33.161312    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:33.840213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:33.840173    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:34.147051 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:34.138935    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:34.829253 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:34.829215    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:35.485181 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:35.485141    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:35.629652 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:35.629602    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:35.821102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:35.807541    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:36.073700 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:36.073607    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:55:36.073948 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:36.073901    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy" containerID="cri-o://c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577" gracePeriod=30
Nov 04 02:55:36.694287 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:55:36.694253    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:55:37.198650 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:37.198624    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:37.452216 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:37.452092    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:37.728224 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:37.722569    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:37.885644 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:37.885607    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:38.033707 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:38.033676    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:38.206899 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:38.206868    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:39.681060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:39.680797    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:40.853201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:40.853166    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:40.996456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:40.996419    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:41.138665 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.138601    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:41.342935 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.342912    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:55:41.414998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.414928    6019 generic.go:356] "Generic (PLEG): container finished" podID="caab526c-8435-496c-973f-3bf4318bb392" containerID="c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577" exitCode=0
Nov 04 02:55:41.414998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.414978    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"
Nov 04 02:55:41.415256 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.415000    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" event={"ID":"caab526c-8435-496c-973f-3bf4318bb392","Type":"ContainerDied","Data":"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"}
Nov 04 02:55:41.415256 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.415053    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7" event={"ID":"caab526c-8435-496c-973f-3bf4318bb392","Type":"ContainerDied","Data":"52730cf25c5aaea5977cb72dfd33fd71a52e034e8decc2b1c725d31187061a1a"}
Nov 04 02:55:41.415256 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.415118    6019 scope.go:122] "RemoveContainer" containerID="c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"
Nov 04 02:55:41.420561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420540    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.420668 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420605    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.420778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420664    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.420778 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420734    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.420890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420812    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.420890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420814    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:55:41.420890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420878    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.421137 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.420993    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.421137 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421027    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.421249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421161    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\") pod \"caab526c-8435-496c-973f-3bf4318bb392\" (UID: \"caab526c-8435-496c-973f-3bf4318bb392\") "
Nov 04 02:55:41.421249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421219    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:55:41.421350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421315    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:55:41.421634 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421533    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:55:41.422004 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.421982    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:55:41.422145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.422057    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.422145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.422098    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.422145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.422113    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.422145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.422126    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.423845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.423586    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:55:41.423845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.423608    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:55:41.423845 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.423762    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "kube-api-access-d6cpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:55:41.424277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.424254    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo" pod "caab526c-8435-496c-973f-3bf4318bb392" (UID: "caab526c-8435-496c-973f-3bf4318bb392"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:55:41.427083 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.427052    6019 scope.go:122] "RemoveContainer" containerID="c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"
Nov 04 02:55:41.427354 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:55:41.427338    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577\": container with ID starting with c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577 not found: ID does not exist" containerID="c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"
Nov 04 02:55:41.427412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.427362    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577"} err="failed to get container status \"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577\": rpc error: code = NotFound desc = could not find container \"c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577\": container with ID starting with c4370511c9fb831f9cef49aaaf14d238ca17ff4db5a6481d462ab90eab7d9577 not found: ID does not exist"
Nov 04 02:55:41.523672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.523637    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caab526c-8435-496c-973f-3bf4318bb392-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.523672 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.523675    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caab526c-8435-496c-973f-3bf4318bb392-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.523870 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.523690    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.523870 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.523704    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caab526c-8435-496c-973f-3bf4318bb392-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.523870 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.523719    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6cpj\" (UniqueName: \"kubernetes.io/projected/caab526c-8435-496c-973f-3bf4318bb392-kube-api-access-d6cpj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:55:41.597719 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.597685    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:41.763299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.763267    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:41.842970 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.842938    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:55:41.852693 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.852670    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7"]
Nov 04 02:55:41.912928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:41.912889    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:42.566394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:42.566362    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:42.982058 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:42.981975    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:43.125646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:43.125604    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caab526c-8435-496c-973f-3bf4318bb392" path="/var/lib/kubelet/pods/caab526c-8435-496c-973f-3bf4318bb392/volumes"
Nov 04 02:55:43.135110 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:43.132275    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:43.314568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:43.314280    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:44.416062 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.416022    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.553271 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.553213    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.575213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.575172    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.575817 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.575783    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:44.627174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.625042    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.628315 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.628281    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.811098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.805095    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.857232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.857184    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.951286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.951221    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:44.960194 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:44.960153    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.132319 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.132204    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": context deadline exceeded"
Nov 04 02:55:45.157108 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.156722    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:55:45.157108 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.156789    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.370245 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.370202    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:55:45.370531 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.370498    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.427670 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.427571    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:55:45.428655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.428634    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:45.572219 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.572170    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.572916 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.572891    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:55:45.600569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.598177    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.634534 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.634506    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:45.664098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.599158    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.679153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.630153    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.679747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.626172    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.680190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.674174    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:55:45.680390 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.680373    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:45.680523 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.680511    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:45.789339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.789197    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.866099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.865900    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.893106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.866303    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.893106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.866621    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:55:45.893106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.866677    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:55:45.953256 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.953161    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:45.971459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.971429    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:55:46.000115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.997123    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.000115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:45.997670    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:55:46.294453 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.294409    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.446620 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.446574    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.590328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.590211    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 02:55:46.659677 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.657607    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.707515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.701381    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.707515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.672120    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.707515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.702996    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:46.925160 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.896354    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 04 02:55:46.983134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.938987    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:55:46.983134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:46.971205    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:47.002199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.002156    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.464110 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.463646    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.597145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.597114    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 02:55:47.693475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.693429    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:47.706370 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.706328    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.707093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.706341    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.707563 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.706381    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.828921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.774243    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:55:47.828921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.774306    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:47.828921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.816042    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:55:47.828921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.816294    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:47.828921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.828190    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:47.970101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.937061    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:47.970101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.939475    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.970101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.939786    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 04 02:55:47.976230 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:47.976192    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:48.004852 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.004166    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 04 02:55:48.230157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.221641    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:48.268316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.268250    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:55:48.268604 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.268553    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:48.638099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.620584    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:48.754099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.716227    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:48.970145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.969275    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Nov 04 02:55:48.970145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.969957    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:48.970397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.970254    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.007130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:48.983186    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.007130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.005354    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.566449 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.566403    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.680499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.680435    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:49.755100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.728419    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.755100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.728467    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:55:49.778201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.728500    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:49.778201 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.728546    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:55:49.879978 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.879865    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.972400 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.972351    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:49.973014 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.972984    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 02:55:49.993997 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:49.993165    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:50.021147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.012019    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:50.021147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.012693    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:50.259320 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.259290    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 02:55:50.552009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.551928    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 02:55:50.567286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.567221    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:50.568807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.567998    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:50.691756 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.690639    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 02:55:50.732965 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.732934    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 02:55:50.773096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.759535    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:55:50.773096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.759226    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 04 02:55:50.773096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.763229    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:50.784094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.782143    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:50.784094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.783137    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:55:50.784094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.783911    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:50.814157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.812981    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:55:50.860098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:50.843443    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 02:55:51.419952 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:51.419916    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:55:51.420180 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:51.419978    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:55:52.464026 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:52.463976    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:56.035567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:56.035536    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:56.179041 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:56.179003    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:56.484352 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:56.484257    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:57.181812 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:57.181776    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:57.390140 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:57.385784    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:57.590490 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:57.590454    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:55:57.775096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:55:57.770647    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:02.316274 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:02.316237    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:02.482083 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:02.482032    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:02.632905 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:02.632813    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:02.779365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:02.779325    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:02.939179 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:02.938910    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:03.762992 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:03.762962    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:05.016807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.016770    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:05.794095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.785626    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:56:05.794095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.791788    6019 cpu_manager.go:414] "containerMap: removing container" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy"
Nov 04 02:56:05.794095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.791816    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy"
Nov 04 02:56:05.794095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.791986    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="caab526c-8435-496c-973f-3bf4318bb392" containerName="istio-proxy"
Nov 04 02:56:05.804394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.804360    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:05.857339 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.857300    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:56:05.931866 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.921674    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:05.931866 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:05.921890    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f73e00ed-openshift-default-dockercfg-6wh2h\""
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.038944    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.052618    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.052672    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.052761    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.052904    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.052958    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdt5q\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.053029    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.053102    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.066149 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.053339    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.095163 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.090705    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:06.158344 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158297    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158555 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158416    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158555 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158501    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158555 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158534    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158601    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158735 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158714    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158839 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158757    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdt5q\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158839 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158812    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.158941 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.158846    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.160331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.159426    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.177843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.163646    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.177843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.163886    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.177843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.164783    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.177843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.165029    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.193092 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.187430    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.210629 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.210560    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.240787 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.240744    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.251748 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.250351    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdt5q\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\") pod \"e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") " pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.447836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.447752    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:06.609210 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:06.609178    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:56:06.888157 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.888129    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:56:06.890162 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:56:06.890127    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9929a12_3a3e_4301_97bf_20209ecfea48.slice/crio-32649a545f69068953badf593fd937760e671c98f3708132ef7097fbbe95c0c6 WatchSource:0}: Error finding container 32649a545f69068953badf593fd937760e671c98f3708132ef7097fbbe95c0c6: Status 404 returned error can't find the container with id 32649a545f69068953badf593fd937760e671c98f3708132ef7097fbbe95c0c6
Nov 04 02:56:06.892825 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.892793    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:06.892930 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.892858    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:06.892930 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:06.892888    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:07.496820 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:07.496787    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerStarted","Data":"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"}
Nov 04 02:56:07.497279 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:07.496847    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerStarted","Data":"32649a545f69068953badf593fd937760e671c98f3708132ef7097fbbe95c0c6"}
Nov 04 02:56:07.618805 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:07.618756    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podStartSLOduration=2.618741537 podStartE2EDuration="2.618741537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:56:07.616285629 +0000 UTC m=+1322.757299036" watchObservedRunningTime="2026-11-04 02:56:07.618741537 +0000 UTC m=+1322.759755015"
Nov 04 02:56:07.695568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:07.695527    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:07.976736 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:07.976684    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:08.448185 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:08.448160    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:08.453713 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:08.453689    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:08.503885 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:08.503854    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:08.504829 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:08.504801    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:56:09.172131 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:09.172092    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:09.316628 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:09.316590    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:09.629749 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:09.629719    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:09.810497 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:09.810455    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:09.957101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:09.956982    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:12.895961 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:12.895913    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:13.040586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.040529    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:13.191085 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.190982    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:13.753726 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.751159    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:56:13.759755 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.759726    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.772120 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.764334    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:56:13.835216 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.835182    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.835372 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.835229    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4nbm\" (UniqueName: \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.865184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.865162    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f73e00ed\""
Nov 04 02:56:13.865346 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.865331    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f73e00ed-dockercfg-srlfg\""
Nov 04 02:56:13.936365 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.936332    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.936779 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.936376    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4nbm\" (UniqueName: \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.937109 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.937092    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:13.997190 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:13.997165    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4nbm\" (UniqueName: \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\") pod \"payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:14.086290 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.086217    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:14.371974 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.371897    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:14.463461 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.463434    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:56:14.465229 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:56:14.465201    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75da892e_5d57_45fd_8f49_d12bd8358531.slice/crio-832138dce9fde1f7ac244af38a3934ff058aaee42f8379abe62f76e6b88236b1 WatchSource:0}: Error finding container 832138dce9fde1f7ac244af38a3934ff058aaee42f8379abe62f76e6b88236b1: Status 404 returned error can't find the container with id 832138dce9fde1f7ac244af38a3934ff058aaee42f8379abe62f76e6b88236b1
Nov 04 02:56:14.579418 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.579387    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" event={"ID":"75da892e-5d57-45fd-8f49-d12bd8358531","Type":"ContainerStarted","Data":"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"}
Nov 04 02:56:14.579561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.579447    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" event={"ID":"75da892e-5d57-45fd-8f49-d12bd8358531","Type":"ContainerStarted","Data":"832138dce9fde1f7ac244af38a3934ff058aaee42f8379abe62f76e6b88236b1"}
Nov 04 02:56:14.579561 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.579469    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:14.697199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:14.697104    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" podStartSLOduration=1.697089487 podStartE2EDuration="1.697089487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:56:14.694195986 +0000 UTC m=+1329.835209395" watchObservedRunningTime="2026-11-04 02:56:14.697089487 +0000 UTC m=+1329.838102891"
Nov 04 02:56:18.017586 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:18.017555    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:22.838177 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.838130    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:56:22.866562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.866532    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.879953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.870353    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.879953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.870467    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.879953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.870606    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8c5l\" (UniqueName: \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.879953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.870658    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.879953 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.877339    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:56:22.951099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.949307    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2c1c4f09-serving-cert\""
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.973105    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8c5l\" (UniqueName: \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.973183    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.973370    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.973454    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.974431    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:22.975348    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-serving-cert: secret "maas-api-e2e-migrate-2c1c4f09-serving-cert" not found
Nov 04 02:56:22.977096 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:22.975409    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls podName:21b136a0-6d62-4f04-9ca6-7e20dd6b48a7 nodeName:}" failed. No retries permitted until 2026-11-04 02:56:23.475388597 +0000 UTC m=+1338.616401993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls") pod "maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7") : secret "maas-api-e2e-migrate-2c1c4f09-serving-cert" not found
Nov 04 02:56:23.007199 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:22.991231    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:23.062527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.062456    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:23.067655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.067596    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8c5l\" (UniqueName: \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:23.638110 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.630943    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:23.638110 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:23.631618    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-serving-cert: secret "maas-api-e2e-migrate-2c1c4f09-serving-cert" not found
Nov 04 02:56:23.681110 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:23.658923    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls podName:21b136a0-6d62-4f04-9ca6-7e20dd6b48a7 nodeName:}" failed. No retries permitted until 2026-11-04 02:56:24.658601442 +0000 UTC m=+1339.799614849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls") pod "maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7") : secret "maas-api-e2e-migrate-2c1c4f09-serving-cert" not found
Nov 04 02:56:23.753347 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.699311    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:23.775096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.735979    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:23.960253 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.960153    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:23.963298 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:23.963267    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:24.048097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.046168    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:24.324098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.323355    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 04 02:56:24.381093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.380536    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:24.415196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.401961    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:24.668356 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.668261    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:24.690907 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.690876    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:24.708350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.708320    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:24.793036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.788210    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:24.793036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.788981    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:56:24.830096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.826411    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:24.830096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:24.826525    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.144107 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.118205    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.442625 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.339157    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.541101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.381165    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 04 02:56:25.541101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.536048    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:56:25.541101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.536133    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:56:25.659011 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.657935    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:56:25.781134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.760091    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.781134 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.760490    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 04 02:56:25.804106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.796855    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.804106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.797221    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": context deadline exceeded"
Nov 04 02:56:25.829099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.828668    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.829099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.829037    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:56:25.949099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.943217    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:25.974158 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:25.959139    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:26.005658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.005162    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:26.031781 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.019653    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:56:26.080142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.078039    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 02:56:26.249097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.224295    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 02:56:26.412790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.388817    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:56:26.412790 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.407125    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:56:26.423626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.422482    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 02:56:26.471632 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:26.471607    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 02:56:27.045459 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:27.045399    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" event={"ID":"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7","Type":"ContainerStarted","Data":"540051d85133395b600507dfd21f585633021e5dcb530e8281bd8ef712fa7e11"}
Nov 04 02:56:28.055464 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:28.055428    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" event={"ID":"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7","Type":"ContainerStarted","Data":"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"}
Nov 04 02:56:28.055849 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:28.055520    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:28.172646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:28.172596    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" podStartSLOduration=5.033026094 podStartE2EDuration="6.172581477s" totalImagesPullingTime="1.139555383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:56:28.171455472 +0000 UTC m=+1343.312468879" watchObservedRunningTime="2026-11-04 02:56:28.172581477 +0000 UTC m=+1343.313594881"
Nov 04 02:56:28.609998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:28.609959    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:33.061431 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:33.061395    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:33.536717 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:33.536662    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:33.679412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:33.679371    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:33.830022 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:33.829812    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:34.066971 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:34.066946    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:56:35.190332 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:35.190299    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:36.643085 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.643033    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 02:56:36.648353 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.648325    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.663539 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:56:36.663512    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:56:36.691631 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.691610    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"]
Nov 04 02:56:36.696141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.696122    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.703004 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.702984    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pm6wh\""
Nov 04 02:56:36.724412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724388    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg4hj\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724513 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724424    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2ppk\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-kube-api-access-t2ppk\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.724588 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724567    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724626 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724615    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724712    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.724779 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724752    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.724779 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724771    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.724862 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724842    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724902 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724879    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724936 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724911    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.724969 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724934    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.724969 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.724954    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.725058 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725046    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.725133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725064    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.725133 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725095    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.725225 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725209    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.725266 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725243    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.725305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.725269    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.747180 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.747154    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 02:56:36.815552 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.815520    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"]
Nov 04 02:56:36.827118 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827083    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.827287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827147    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827191    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827254    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg4hj\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.827287 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827281    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2ppk\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-kube-api-access-t2ppk\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827525 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827429    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.827525 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827516    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.827639 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827599    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827639    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827697 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827666    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.827830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827699    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.827830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827729    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828036 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.827993    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.828202 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828167    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828275 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828224    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828337 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828271    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828396 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828360    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828396 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828386    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.828499 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828419    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.828604 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828577    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828767 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828710    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828882 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828836    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.828987 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.828951    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.829371 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.829345    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.829475 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.829442    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.829622 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.829600    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.829887 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.829857    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.830056 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.830032    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.832798 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.832774    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.833497 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.833471    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.833978 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.833953    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.834125 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.834064    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.890975 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.890947    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.891128 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.890951    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.891270 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.891248    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2ppk\" (UniqueName: \"kubernetes.io/projected/4afac37a-d227-4dd0-8f7d-b7c276c4974c-kube-api-access-t2ppk\") pod \"maas-default-gateway-openshift-default-76597d6855-74hkj\" (UID: \"4afac37a-d227-4dd0-8f7d-b7c276c4974c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:36.891328 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.891251    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg4hj\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\") pod \"maas-default-gateway-openshift-default-76597d6855-xrlzk\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:36.964755 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:36.964680    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:37.012361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.012318    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:37.398730 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.398701    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 02:56:37.399568 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:56:37.399544    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556dc3ec_4888_4b5d_909b_6a5023593b55.slice/crio-ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22 WatchSource:0}: Error finding container ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22: Status 404 returned error can't find the container with id ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22
Nov 04 02:56:37.402029 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.401997    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:37.402169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.402090    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:37.402169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.402138    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:37.452534 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.452511    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"]
Nov 04 02:56:37.452893 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:56:37.452870    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afac37a_d227_4dd0_8f7d_b7c276c4974c.slice/crio-bc8f9bd3125fd96dd1bc8ac31ce34648d988656c820bd5dc99d649caddbc369f WatchSource:0}: Error finding container bc8f9bd3125fd96dd1bc8ac31ce34648d988656c820bd5dc99d649caddbc369f: Status 404 returned error can't find the container with id bc8f9bd3125fd96dd1bc8ac31ce34648d988656c820bd5dc99d649caddbc369f
Nov 04 02:56:37.455520 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.455491    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:37.455623 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.455566    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:37.455623 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:37.455613    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:56:38.145288 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.145254    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" event={"ID":"4afac37a-d227-4dd0-8f7d-b7c276c4974c","Type":"ContainerStarted","Data":"b1415dd4a1716d78d763f845542590f091b16e48a8018a4c47d1b0029e290654"}
Nov 04 02:56:38.145728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.145311    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" event={"ID":"4afac37a-d227-4dd0-8f7d-b7c276c4974c","Type":"ContainerStarted","Data":"bc8f9bd3125fd96dd1bc8ac31ce34648d988656c820bd5dc99d649caddbc369f"}
Nov 04 02:56:38.146944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.146921    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" event={"ID":"556dc3ec-4888-4b5d-909b-6a5023593b55","Type":"ContainerStarted","Data":"8e5619dc6b0e5e07f8e5d9317543a3b3a9b35f604f0fed652831cb33bc62df4c"}
Nov 04 02:56:38.147064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.146962    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" event={"ID":"556dc3ec-4888-4b5d-909b-6a5023593b55","Type":"ContainerStarted","Data":"ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22"}
Nov 04 02:56:38.268877 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.268805    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podStartSLOduration=2.268781969 podStartE2EDuration="2.268781969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:56:38.264640076 +0000 UTC m=+1353.405653485" watchObservedRunningTime="2026-11-04 02:56:38.268781969 +0000 UTC m=+1353.409795377"
Nov 04 02:56:38.388804 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.388744    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podStartSLOduration=2.388723044 podStartE2EDuration="2.388723044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:56:38.384107141 +0000 UTC m=+1353.525120551" watchObservedRunningTime="2026-11-04 02:56:38.388723044 +0000 UTC m=+1353.529736452"
Nov 04 02:56:38.670147 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.670109    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:38.976950 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.972454    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:38.988095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.979585    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:38.988095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:38.979641    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:39.013445 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:39.013409    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:39.014954 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:39.014925    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:39.015018 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:39.014985    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:39.971097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:39.968719    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:39.971097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:39.968815    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:40.021341 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:40.021307    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:40.021601 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:40.021572    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:40.970096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:40.967430    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:40.970096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:40.967500    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:41.012989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:41.012953    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:41.013384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:41.013019    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:41.971507 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:41.971467    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:41.971808 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:41.971779    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:42.012800 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:42.012766    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:42.013039 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:42.013015    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:42.967508 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:42.967465    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:42.967803 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:42.967774    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:43.012611 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:43.012576    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:43.012770 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:43.012621    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:43.594928 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:43.594885    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:43.974671 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:43.974580    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:43.975124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:43.975089    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:44.013184 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:44.013147    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:44.013349 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:44.013210    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:44.966012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:44.965977    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:56:44.966470 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:44.966043    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:56:45.013057 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:45.013020    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:56:45.013232 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:45.013106    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:56:46.802800 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.802766    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:46.803327 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.803307    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:46.804320 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.804300    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj"
Nov 04 02:56:46.815005 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.814983    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:46.815312 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.815288    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:46.816062 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:46.816047    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 02:56:48.487294 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:48.487265    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:53.398094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:53.398039    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:56:58.391450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:56:58.391383    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:02.484754 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:02.484712    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:02.941844 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:02.941811    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:57:02.946427 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:02.946409    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.000060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:02.999937    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f73e00ed-serving-cert\""
Nov 04 02:57:03.055366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.055332    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:57:03.071121 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.071092    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8xzz\" (UniqueName: \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.071305 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.071160    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.071412 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.071388    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.071477 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.071434    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.132308 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.132277    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:03.173138 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.173066    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8xzz\" (UniqueName: \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.173316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.173174    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.173316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.173225    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.173316 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.173256    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.173316 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:57:03.173305    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-serving-cert: secret "maas-api-e2e-ait-f73e00ed-serving-cert" not found
Nov 04 02:57:03.173507 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:57:03.173357    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls podName:d44e9f3d-ee0d-4806-a1ca-b012dafe2451 nodeName:}" failed. No retries permitted until 2026-11-04 02:57:03.673343391 +0000 UTC m=+1378.814356776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls") pod "maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" (UID: "d44e9f3d-ee0d-4806-a1ca-b012dafe2451") : secret "maas-api-e2e-ait-f73e00ed-serving-cert" not found
Nov 04 02:57:03.173937 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.173917    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.175825 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.175806    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.244450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.244374    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8xzz\" (UniqueName: \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.307621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.307581    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:03.577383 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.577346    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:03.701269 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.700751    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.716404 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.716373    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") pod \"maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:03.862300 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:03.862220    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:04.240777 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:04.240753    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:57:04.241936 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:57:04.241907    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44e9f3d_ee0d_4806_a1ca_b012dafe2451.slice/crio-bf9b251c023b19a579bc12cd8fdfcc3820926cfea8efc9faaff9f6172d760593 WatchSource:0}: Error finding container bf9b251c023b19a579bc12cd8fdfcc3820926cfea8efc9faaff9f6172d760593: Status 404 returned error can't find the container with id bf9b251c023b19a579bc12cd8fdfcc3820926cfea8efc9faaff9f6172d760593
Nov 04 02:57:04.374037 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:04.374002    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" event={"ID":"d44e9f3d-ee0d-4806-a1ca-b012dafe2451","Type":"ContainerStarted","Data":"bf9b251c023b19a579bc12cd8fdfcc3820926cfea8efc9faaff9f6172d760593"}
Nov 04 02:57:05.383836 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:05.383804    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" event={"ID":"d44e9f3d-ee0d-4806-a1ca-b012dafe2451","Type":"ContainerStarted","Data":"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"}
Nov 04 02:57:05.384265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:05.383893    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:05.502394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:05.502345    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" podStartSLOduration=2.564886591 podStartE2EDuration="3.502331262s" totalImagesPullingTime="937.444671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:57:05.49968977 +0000 UTC m=+1380.640703180" watchObservedRunningTime="2026-11-04 02:57:05.502331262 +0000 UTC m=+1380.643344670"
Nov 04 02:57:06.608299 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:57:06.608264    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:57:08.533986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:08.533933    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:11.395843 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:11.395813    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:57:13.517986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:13.517956    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:18.392478 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:18.392427    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:22.963530 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:22.963496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:27.472745 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:27.472706    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:27.619819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:27.619787    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:28.275614 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:28.275580    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:33.418213 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:33.418158    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:33.569117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:33.569047    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:35.542105 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:35.542035    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:36.604191 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:57:36.604156    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:57:37.822748 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:37.822715    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:37.970551 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:37.970516    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:38.356129 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:38.356096    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:38.699297 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:38.699197    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:44.177009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:44.176968    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:49.518585 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:49.518545    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:49.807784 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:49.807704    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:51.785567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.785539    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.909549    6019 generic.go:356] "Generic (PLEG): container finished" podID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerID="fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4" exitCode=0
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.909642    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerDied","Data":"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"}
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.916367    6019 scope.go:122] "RemoveContainer" containerID="fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.918795    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.918891    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:57:51.953369 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.918937    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:57:51.993549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:51.993189    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:52.966098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:52.964613    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerStarted","Data":"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"}
Nov 04 02:57:53.451098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:53.448392    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:57:53.462098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:53.457429    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:53.462098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:53.457508    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:53.510218 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:53.510181    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:57:54.450124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:54.448418    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:54.450124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:54.448492    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:54.518798 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:54.518770    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:55.457444 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:55.457409    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:55.458049 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:55.458018    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:55.778411 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:55.778378    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:56.471099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:56.470806    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:56.471099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:56.470874    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:57.455100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:57.453410    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:57.455100 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:57.453468    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:58.448591 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:58.448550    6019 patch_prober.go:29] interesting pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 02:57:58.448997 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:58.448627    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 02:57:59.366922 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:59.366887    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:57:59.486595 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:59.486565    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:57:59.492527 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:57:59.492488    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:58:04.377278 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:04.377238    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:06.604887 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:06.604850    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:58:09.671709 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:09.671668    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:14.194417 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.194387    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:58:14.194853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.194573    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" podUID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" containerName="maas-api" containerID="cri-o://621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9" gracePeriod=30
Nov 04 02:58:14.334164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.334116    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:14.757837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.757391    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:58:14.822252 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.822211    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") pod \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") "
Nov 04 02:58:14.822931 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.822904    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\" (UniqueName: \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\") pod \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") "
Nov 04 02:58:14.823568 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.823531    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\") pod \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") "
Nov 04 02:58:14.836487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.836338    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca" pod "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:58:14.836881 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.836861    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\") pod \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\" (UID: \"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7\") "
Nov 04 02:58:14.838312 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.838291    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:14.856194 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.856156    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l" pod "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7"). InnerVolumeSpecName "kube-api-access-c8c5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:58:14.857238 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.857213    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls" pod "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:58:14.857476 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.857451    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs" pod "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" (UID: "21b136a0-6d62-4f04-9ca6-7e20dd6b48a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:58:14.958159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.953600    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:14.958159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.953634    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:14.958159 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:14.953650    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8c5l\" (UniqueName: \"kubernetes.io/projected/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7-kube-api-access-c8c5l\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:15.135594 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.135556    6019 generic.go:356] "Generic (PLEG): container finished" podID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" containerID="621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9" exitCode=0
Nov 04 02:58:15.135594 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.135605    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"
Nov 04 02:58:15.135801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.135613    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" event={"ID":"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7","Type":"ContainerDied","Data":"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"}
Nov 04 02:58:15.135801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.135668    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8" event={"ID":"21b136a0-6d62-4f04-9ca6-7e20dd6b48a7","Type":"ContainerDied","Data":"540051d85133395b600507dfd21f585633021e5dcb530e8281bd8ef712fa7e11"}
Nov 04 02:58:15.135801 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.135698    6019 scope.go:122] "RemoveContainer" containerID="621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"
Nov 04 02:58:15.145978 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.145957    6019 scope.go:122] "RemoveContainer" containerID="621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"
Nov 04 02:58:15.146409 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:15.146386    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9\": container with ID starting with 621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9 not found: ID does not exist" containerID="621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"
Nov 04 02:58:15.146495 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.146415    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9"} err="failed to get container status \"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9\": rpc error: code = NotFound desc = could not find container \"621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9\": container with ID starting with 621bb9444b8b9c9e440c0ab079bc166bfb56b952882aed6d16cbad0bc63819e9 not found: ID does not exist"
Nov 04 02:58:15.302680 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.302646    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:58:15.351982 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.351950    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2c1c4f09-5d846ccfd6-4r6x8"]
Nov 04 02:58:15.887210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:15.887178    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:16.152303 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:16.152219    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:16.302666 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:16.300861    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:16.725379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:16.725350    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:17.022765 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.022727    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" path="/var/lib/kubelet/pods/21b136a0-6d62-4f04-9ca6-7e20dd6b48a7/volumes"
Nov 04 02:58:17.327128 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.326980    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:17.894852 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.894815    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 02:58:17.895921 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.895898    6019 cpu_manager.go:414] "containerMap: removing container" podUID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" containerName="maas-api"
Nov 04 02:58:17.895979 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.895927    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" containerName="maas-api"
Nov 04 02:58:17.896146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.896132    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="21b136a0-6d62-4f04-9ca6-7e20dd6b48a7" containerName="maas-api"
Nov 04 02:58:17.901769 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.901747    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.937656 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.937616    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.937817 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.937670    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.937817 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.937730    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.937817 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.937802    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.937989 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.937848    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.938062 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.938042    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.938170 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.938112    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8lnq\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.938336 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.938309    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.938389 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.938372    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:17.954799 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.954384    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4d5a0200-openshift-default-dockercfg-zpssh\""
Nov 04 02:58:17.989436 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:17.989394    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:18.012733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.012704    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039529    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039596    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8lnq\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039659    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039705    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039773    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039818    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039880    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.039958    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.040007    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.041104 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.040727    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.060153 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.060120    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.060929 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.060901    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.061702 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.061678    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.062151 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.062131    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.435220 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.435124    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.480549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.480509    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.490094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.486545    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.507094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.504580    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:18.516112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.516022    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8lnq\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\") pod \"e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") " pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:18.546450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:18.546178    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:19.138431 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.138383    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 02:58:19.140570 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:58:19.140532    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9750f6ab_8091_494d_8228_b29f40c09080.slice/crio-8e0e17897be845c517d63e379488435ce6ded0cb349e00697f787af77790a54f WatchSource:0}: Error finding container 8e0e17897be845c517d63e379488435ce6ded0cb349e00697f787af77790a54f: Status 404 returned error can't find the container with id 8e0e17897be845c517d63e379488435ce6ded0cb349e00697f787af77790a54f
Nov 04 02:58:19.144573 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.144534    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:19.144689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.144622    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:19.144689 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.144663    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:19.526102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.525536    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerStarted","Data":"50429ee1550b3c21b04ae167f25c6e35c96259568c1c183b7de4ccec5bfb1b74"}
Nov 04 02:58:19.526102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.525588    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerStarted","Data":"8e0e17897be845c517d63e379488435ce6ded0cb349e00697f787af77790a54f"}
Nov 04 02:58:19.547590 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.547504    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:19.686094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:19.676465    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podStartSLOduration=2.676443758 podStartE2EDuration="2.676443758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:58:19.648480049 +0000 UTC m=+1454.789493457" watchObservedRunningTime="2026-11-04 02:58:19.676443758 +0000 UTC m=+1454.817457171"
Nov 04 02:58:20.224101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.223951    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:58:20.404622 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.404325    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:20.643192 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.642463    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:20.824581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.816765    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:20.824581 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.818732    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:20.862259 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.861966    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:58:20.981253 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:20.981222    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:21.677301 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:21.677252    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:21.859654 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:21.859617    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:22.050379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:22.050347    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:22.639119 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:22.638997    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:22.951823 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:22.951717    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:23.096680 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:23.096606    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:23.272851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:23.272814    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:23.852322 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:23.852285    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:25.225008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.224950    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" containerName="payload-pre-processing" containerID="cri-o://0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3" gracePeriod=25
Nov 04 02:58:25.284451 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.284422    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:25.432144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.431995    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:25.623819 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.623781    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:25.782996 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.782866    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:26.002183 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:25.999453    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:58:26.091113 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.088957    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\" (UniqueName: \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\") pod \"75da892e-5d57-45fd-8f49-d12bd8358531\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") "
Nov 04 02:58:26.091113 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.089412    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\") pod \"75da892e-5d57-45fd-8f49-d12bd8358531\" (UID: \"75da892e-5d57-45fd-8f49-d12bd8358531\") "
Nov 04 02:58:26.094364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.092950    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume" pod "75da892e-5d57-45fd-8f49-d12bd8358531" (UID: "75da892e-5d57-45fd-8f49-d12bd8358531"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:58:26.222206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.215353    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm" pod "75da892e-5d57-45fd-8f49-d12bd8358531" (UID: "75da892e-5d57-45fd-8f49-d12bd8358531"). InnerVolumeSpecName "kube-api-access-h4nbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:58:26.226747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.225343    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75da892e-5d57-45fd-8f49-d12bd8358531-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:26.226747 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.225376    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4nbm\" (UniqueName: \"kubernetes.io/projected/75da892e-5d57-45fd-8f49-d12bd8358531-kube-api-access-h4nbm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:26.314178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.313996    6019 generic.go:356] "Generic (PLEG): container finished" podID="75da892e-5d57-45fd-8f49-d12bd8358531" containerID="0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3" exitCode=0
Nov 04 02:58:26.314608 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.314376    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" event={"ID":"75da892e-5d57-45fd-8f49-d12bd8358531","Type":"ContainerDied","Data":"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"}
Nov 04 02:58:26.314608 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.314450    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" event={"ID":"75da892e-5d57-45fd-8f49-d12bd8358531","Type":"ContainerDied","Data":"832138dce9fde1f7ac244af38a3934ff058aaee42f8379abe62f76e6b88236b1"}
Nov 04 02:58:26.314608 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.314489    6019 scope.go:122] "RemoveContainer" containerID="0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"
Nov 04 02:58:26.314830 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.314634    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"
Nov 04 02:58:26.487314 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.476440    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:58:26.507097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.505565    6019 scope.go:122] "RemoveContainer" containerID="0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"
Nov 04 02:58:26.507910 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:26.507682    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3\": container with ID starting with 0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3 not found: ID does not exist" containerID="0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"
Nov 04 02:58:26.507910 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.507727    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3"} err="failed to get container status \"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3\": rpc error: code = NotFound desc = could not find container \"0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3\": container with ID starting with 0639cab011e0e4c829b010011eda06e7f7bac253a384f3276cd494e8d8fd88e3 not found: ID does not exist"
Nov 04 02:58:26.621046 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.620198    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.77:50052\" within 1s: context deadline exceeded"
Nov 04 02:58:26.621046 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:58:26.620520    6019 logging.go:55] [core] [Channel #539 SubChannel #540]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.77:50052", ServerName: "10.132.0.77:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.77:50052: operation was canceled"
Nov 04 02:58:26.624728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.624592    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:58:26.629272 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.629239    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv"]
Nov 04 02:58:26.983012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:26.982934    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" path="/var/lib/kubelet/pods/75da892e-5d57-45fd-8f49-d12bd8358531/volumes"
Nov 04 02:58:28.402887 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:28.402811    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:29.074258 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:29.074226    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:29.242487 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:29.242460    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:29.387310 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:29.387229    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:32.356633 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:32.356589    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:32.547966 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:32.547912    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:33.702968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:33.702922    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:33.866658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:33.866606    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:35.900998 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:35.900970    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:36.604983 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:36.604951    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:58:37.677926 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.677886    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 02:58:37.678783 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.678757    6019 cpu_manager.go:414] "containerMap: removing container" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" containerName="payload-pre-processing"
Nov 04 02:58:37.678783 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.678782    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" containerName="payload-pre-processing"
Nov 04 02:58:37.678985 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.678948    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="75da892e-5d57-45fd-8f49-d12bd8358531" containerName="payload-pre-processing"
Nov 04 02:58:37.683838 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.683821    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.838722 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.838687    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.838888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.838742    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.838888 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.838850    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.838968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.838914    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.838968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.838943    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.839112 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.839057    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.839175 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.839161    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.839223 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.839209    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mggkm\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.839386 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.839373    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.884716 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.884687    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 02:58:37.941383 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941299    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941401    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941465    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mggkm\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941594    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941625    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941667    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941728    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941739    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941745    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941825    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.941968 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.941862    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.942141 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.942116    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.942343 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.942325    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.942655 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.942630    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.944514 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.944495    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:37.945646 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:37.945624    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:38.002760 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.002728    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mggkm\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:38.002890 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.002737    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-rlhz6\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:38.298268 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.298232    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:38.602853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.602771    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:38.763757 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.763599    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 02:58:38.793268 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:58:38.793239    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc093e4d_e9ca_4bc2_8bf3_9c95dc748771.slice/crio-ab74886c2facbf8d5ee8ea84417c4bdceec08d6467784263bd424a05ccd98be2 WatchSource:0}: Error finding container ab74886c2facbf8d5ee8ea84417c4bdceec08d6467784263bd424a05ccd98be2: Status 404 returned error can't find the container with id ab74886c2facbf8d5ee8ea84417c4bdceec08d6467784263bd424a05ccd98be2
Nov 04 02:58:38.795886 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.795852    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:38.795986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.795936    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:38.795986 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:38.795977    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:58:39.435467 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:39.435377    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" event={"ID":"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771","Type":"ContainerStarted","Data":"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"}
Nov 04 02:58:39.435467 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:39.435443    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" event={"ID":"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771","Type":"ContainerStarted","Data":"ab74886c2facbf8d5ee8ea84417c4bdceec08d6467784263bd424a05ccd98be2"}
Nov 04 02:58:39.604773 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:39.604660    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podStartSLOduration=2.604641549 podStartE2EDuration="2.604641549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:58:39.604542062 +0000 UTC m=+1474.745555468" watchObservedRunningTime="2026-11-04 02:58:39.604641549 +0000 UTC m=+1474.745654979"
Nov 04 02:58:40.299254 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:40.299218    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:40.300694 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:40.300667    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:40.300825 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:40.300713    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:41.298739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:41.298705    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:41.298917 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:41.298756    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:42.299621 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:42.299583    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:42.300217 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:42.300190    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:43.299397 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:43.299366    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:43.299645 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:43.299418    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:43.612298 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:43.612180    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:44.299248 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:44.299213    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:44.299425 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:44.299265    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:45.298769 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:45.298731    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:45.299208 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:45.298787    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:46.299017 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:46.298980    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 02:58:46.299424 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:46.299037    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 02:58:47.606093 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.606049    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:47.606977 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.606950    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:47.608225 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.608193    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 02:58:47.762294 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.762262    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:58:47.762482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.762459    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" podUID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" containerName="maas-api" containerID="cri-o://54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06" gracePeriod=30
Nov 04 02:58:47.872309 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:47.872231    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:48.462108 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.456263    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:58:48.565176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.560893    6019 generic.go:356] "Generic (PLEG): container finished" podID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" containerID="54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06" exitCode=0
Nov 04 02:58:48.565176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.562008    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"
Nov 04 02:58:48.565176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.562171    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" event={"ID":"d44e9f3d-ee0d-4806-a1ca-b012dafe2451","Type":"ContainerDied","Data":"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"}
Nov 04 02:58:48.565176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.562217    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm" event={"ID":"d44e9f3d-ee0d-4806-a1ca-b012dafe2451","Type":"ContainerDied","Data":"bf9b251c023b19a579bc12cd8fdfcc3820926cfea8efc9faaff9f6172d760593"}
Nov 04 02:58:48.565176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.562246    6019 scope.go:122] "RemoveContainer" containerID="54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"
Nov 04 02:58:48.581448 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.580619    6019 scope.go:122] "RemoveContainer" containerID="54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"
Nov 04 02:58:48.581448 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:48.581031    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06\": container with ID starting with 54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06 not found: ID does not exist" containerID="54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"
Nov 04 02:58:48.581448 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.581088    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06"} err="failed to get container status \"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06\": rpc error: code = NotFound desc = could not find container \"54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06\": container with ID starting with 54af4a6a4339a3b404deda398d9e75e984b34d043c82e1c97b3eb91fb331ca06 not found: ID does not exist"
Nov 04 02:58:48.593096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.592768    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\" (UniqueName: \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\") pod \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") "
Nov 04 02:58:48.593096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.592925    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\") pod \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") "
Nov 04 02:58:48.593096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.593045    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\") pod \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") "
Nov 04 02:58:48.593416 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.593220    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") pod \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\" (UID: \"d44e9f3d-ee0d-4806-a1ca-b012dafe2451\") "
Nov 04 02:58:48.597096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.596504    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca" pod "d44e9f3d-ee0d-4806-a1ca-b012dafe2451" (UID: "d44e9f3d-ee0d-4806-a1ca-b012dafe2451"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:58:48.605166 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.605108    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs" pod "d44e9f3d-ee0d-4806-a1ca-b012dafe2451" (UID: "d44e9f3d-ee0d-4806-a1ca-b012dafe2451"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:58:48.606856 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.606815    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz" pod "d44e9f3d-ee0d-4806-a1ca-b012dafe2451" (UID: "d44e9f3d-ee0d-4806-a1ca-b012dafe2451"). InnerVolumeSpecName "kube-api-access-g8xzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:58:48.611470 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.611246    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls" pod "d44e9f3d-ee0d-4806-a1ca-b012dafe2451" (UID: "d44e9f3d-ee0d-4806-a1ca-b012dafe2451"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:58:48.696008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.695902    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8xzz\" (UniqueName: \"kubernetes.io/projected/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-kube-api-access-g8xzz\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:48.696008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.695935    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:48.696008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.695949    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:48.696008 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:48.695959    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d44e9f3d-ee0d-4806-a1ca-b012dafe2451-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:49.062549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:49.062506    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:58:49.066670 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:49.066638    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f73e00ed-79ff65bf-q97xm"]
Nov 04 02:58:49.442725 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:49.442664    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:49.698385 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:49.698298    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:49.850096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:49.848610    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:50.541970 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:50.541930    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:50.547472 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:50.547439    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:58:50.547714 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:50.547688    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy" containerID="cri-o://1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8" gracePeriod=30
Nov 04 02:58:50.709099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:50.707648    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:50.998728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:50.998687    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" path="/var/lib/kubelet/pods/d44e9f3d-ee0d-4806-a1ca-b012dafe2451/volumes"
Nov 04 02:58:51.433061 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:51.432972    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:52.008861 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:52.008830    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:52.269758 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:52.269669    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:55.084280 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.084241    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:55.828082 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.828007    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:58:55.900916 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.900882    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901102 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.900973    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901176 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901154    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:58:55.901222 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901189    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901261 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901220    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901261 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901240    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:58:55.901322 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901273    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901322 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901309    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901384 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901335    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901435 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901419    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901489 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901454    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\") pod \"e9929a12-3a3e-4301-97bf-20209ecfea48\" (UID: \"e9929a12-3a3e-4301-97bf-20209ecfea48\") "
Nov 04 02:58:55.901565 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901540    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:58:55.901848 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901812    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:58:55.901848 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.901841    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:58:55.902456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.902430    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:55.902567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.902462    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9929a12-3a3e-4301-97bf-20209ecfea48-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:55.902567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.902476    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:55.902567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.902491    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:55.902567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.902504    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:55.903761 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.903736    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "kube-api-access-xdt5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:58:55.903833 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.903781    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:58:55.904012 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.903984    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:58:55.904142 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:55.904092    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token" pod "e9929a12-3a3e-4301-97bf-20209ecfea48" (UID: "e9929a12-3a3e-4301-97bf-20209ecfea48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:58:56.003853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.003819    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:56.003853 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.003850    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdt5q\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-kube-api-access-xdt5q\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:56.004064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.003863    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:56.004064 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.003871    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9929a12-3a3e-4301-97bf-20209ecfea48-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:58:56.497106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.497060    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:56.668331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.668293    6019 generic.go:356] "Generic (PLEG): container finished" podID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerID="1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8" exitCode=0
Nov 04 02:58:56.668486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.668354    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"
Nov 04 02:58:56.668486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.668375    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerDied","Data":"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"}
Nov 04 02:58:56.668486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.668429    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l" event={"ID":"e9929a12-3a3e-4301-97bf-20209ecfea48","Type":"ContainerDied","Data":"32649a545f69068953badf593fd937760e671c98f3708132ef7097fbbe95c0c6"}
Nov 04 02:58:56.668486 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.668453    6019 scope.go:122] "RemoveContainer" containerID="1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"
Nov 04 02:58:56.674839 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.674815    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:56.680372 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.680353    6019 scope.go:122] "RemoveContainer" containerID="fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"
Nov 04 02:58:56.689452 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.689436    6019 scope.go:122] "RemoveContainer" containerID="1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"
Nov 04 02:58:56.689695 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:56.689678    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8\": container with ID starting with 1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8 not found: ID does not exist" containerID="1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"
Nov 04 02:58:56.689744 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.689706    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8"} err="failed to get container status \"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8\": rpc error: code = NotFound desc = could not find container \"1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8\": container with ID starting with 1342d27ec858185441317c6938474740a883cb693726514c77b5964a062744d8 not found: ID does not exist"
Nov 04 02:58:56.689744 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.689724    6019 scope.go:122] "RemoveContainer" containerID="fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"
Nov 04 02:58:56.689964 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:58:56.689934    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4\": container with ID starting with fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4 not found: ID does not exist" containerID="fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"
Nov 04 02:58:56.690004 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.689969    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4"} err="failed to get container status \"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4\": rpc error: code = NotFound desc = could not find container \"fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4\": container with ID starting with fed68ea77fc60e6c4074e8b4de4882541f2fcef1f9569b740d0ff1ca6ea04af4 not found: ID does not exist"
Nov 04 02:58:56.820866 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.820834    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:58:56.833649 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.833622    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:58:56.883661 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.883631    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l"]
Nov 04 02:58:56.991474 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:56.991444    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" path="/var/lib/kubelet/pods/e9929a12-3a3e-4301-97bf-20209ecfea48/volumes"
Nov 04 02:58:57.150056 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:58:57.149974    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:00.399569 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:00.399514    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:02.252466 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.252427    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:02.467164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.467060    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:02.671250 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671173    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 02:59:02.671837 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671824    6019 cpu_manager.go:414] "containerMap: removing container" podUID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" containerName="maas-api"
Nov 04 02:59:02.671877 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671839    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" containerName="maas-api"
Nov 04 02:59:02.671912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671885    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.671912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671891    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.671912 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671908    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.672009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671913    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.672009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.671993    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.672009 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.672003    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9929a12-3a3e-4301-97bf-20209ecfea48" containerName="istio-proxy"
Nov 04 02:59:02.672123 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.672010    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="d44e9f3d-ee0d-4806-a1ca-b012dafe2451" containerName="maas-api"
Nov 04 02:59:02.677394 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.677375    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:02.732284 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.732246    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 02:59:02.820098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.779343    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-072ece-dockercfg-d25gj\""
Nov 04 02:59:02.820362 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.783829    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:02.820549 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.820527    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntnmq\" (UniqueName: \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:02.879094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.875982    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-072ece\""
Nov 04 02:59:02.979180 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.961564    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:03.011276 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.010899    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntnmq\" (UniqueName: \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:03.012310 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:02.982706    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:03.046983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.046846    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 02:59:03.054248 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.054212    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 02:59:03.054553 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.054534    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.149277 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.149238    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw9kk\" (UniqueName: \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.149560 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.149539    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.151366 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.151338    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntnmq\" (UniqueName: \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\") pod \"payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:03.251743 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.251710    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw9kk\" (UniqueName: \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.251927 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.251778    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.252538 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.252510    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.292705 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.292675    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:03.312493 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.312469    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw9kk\" (UniqueName: \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\") pod \"payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.464980 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.464949    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:03.677761 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.677735    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 02:59:03.678908 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:03.678876    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4af128_f2dd_4930_bb1f_baf8339f2bd4.slice/crio-c2791450cfaf25caa95d87f7ff4a43ba40b79a1f24fa976ff2950acace2df530 WatchSource:0}: Error finding container c2791450cfaf25caa95d87f7ff4a43ba40b79a1f24fa976ff2950acace2df530: Status 404 returned error can't find the container with id c2791450cfaf25caa95d87f7ff4a43ba40b79a1f24fa976ff2950acace2df530
Nov 04 02:59:03.715318 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.715284    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" event={"ID":"ac4af128-f2dd-4930-bb1f-baf8339f2bd4","Type":"ContainerStarted","Data":"c2791450cfaf25caa95d87f7ff4a43ba40b79a1f24fa976ff2950acace2df530"}
Nov 04 02:59:03.845085 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:03.845035    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 02:59:03.846247 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:03.846221    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f1e61b_dd6b_41b3_a84d_0aadb4a05c4f.slice/crio-1ff684a9ba13638ed5c302332ec6787e38b1cde238beac80c86b748d32a078b5 WatchSource:0}: Error finding container 1ff684a9ba13638ed5c302332ec6787e38b1cde238beac80c86b748d32a078b5: Status 404 returned error can't find the container with id 1ff684a9ba13638ed5c302332ec6787e38b1cde238beac80c86b748d32a078b5
Nov 04 02:59:04.723603 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.723563    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" event={"ID":"ac4af128-f2dd-4930-bb1f-baf8339f2bd4","Type":"ContainerStarted","Data":"1dcd8be61b57327bdf4ed70d6f2d4f21bad491e1caf80917d03b08fe51c00bc5"}
Nov 04 02:59:04.724060 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.723637    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:04.725249 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.725227    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" event={"ID":"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f","Type":"ContainerStarted","Data":"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"}
Nov 04 02:59:04.725359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.725269    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" event={"ID":"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f","Type":"ContainerStarted","Data":"1ff684a9ba13638ed5c302332ec6787e38b1cde238beac80c86b748d32a078b5"}
Nov 04 02:59:04.725359 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.725298    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:04.839282 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.839234    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" podStartSLOduration=2.839220284 podStartE2EDuration="2.839220284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:04.837172694 +0000 UTC m=+1499.978186100" watchObservedRunningTime="2026-11-04 02:59:04.839220284 +0000 UTC m=+1499.980233691"
Nov 04 02:59:04.987156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:04.983666    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" podStartSLOduration=2.983645499 podStartE2EDuration="2.983645499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:04.969470877 +0000 UTC m=+1500.110484285" watchObservedRunningTime="2026-11-04 02:59:04.983645499 +0000 UTC m=+1500.124658906"
Nov 04 02:59:05.065101 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.064292    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 02:59:05.098187 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.073464    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.105947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.104873    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.105947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.104978    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.105947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.105144    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.105947 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.105247    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb2rg\" (UniqueName: \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.110413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.110386    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 02:59:05.131203 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.131180    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4d5a0200-serving-cert\""
Nov 04 02:59:05.165878 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:05.165824    6019 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67610923_55e2_4444_918e_ffeb7bebed07.slice\": RecentStats: unable to find data in memory cache]"
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.273601    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.273706    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.273841    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.273918    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb2rg\" (UniqueName: \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:05.274655    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-serving-cert: secret "maas-api-e2e-ait-4d5a0200-serving-cert" not found
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:05.274716    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls podName:67610923-55e2-4444-918e-ffeb7bebed07 nodeName:}" failed. No retries permitted until 2026-11-04 02:59:05.774697292 +0000 UTC m=+1500.915710701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls") pod "maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" (UID: "67610923-55e2-4444-918e-ffeb7bebed07") : secret "maas-api-e2e-ait-4d5a0200-serving-cert" not found
Nov 04 02:59:05.277025 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.275624    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.286178 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.283764    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:05.306733 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.306694    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.394106 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.393405    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb2rg\" (UniqueName: \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.443540 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.441613    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:05.590210 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.586951    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:05.812722 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.812681    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.815437 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.815417    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") pod \"maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:05.852960 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.852840    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:59:05.866787 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:05.866766    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 02:59:06.029745 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:06.029709    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:06.411501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:06.411478    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 02:59:06.413472 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:06.413429    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67610923_55e2_4444_918e_ffeb7bebed07.slice/crio-58fc2f233418a98f400d2bcad371a4bd17cf152d698d7e7020f0b3173d396496 WatchSource:0}: Error finding container 58fc2f233418a98f400d2bcad371a4bd17cf152d698d7e7020f0b3173d396496: Status 404 returned error can't find the container with id 58fc2f233418a98f400d2bcad371a4bd17cf152d698d7e7020f0b3173d396496
Nov 04 02:59:06.608205 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:06.608169    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:59:06.740443 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:06.740353    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" event={"ID":"67610923-55e2-4444-918e-ffeb7bebed07","Type":"ContainerStarted","Data":"58fc2f233418a98f400d2bcad371a4bd17cf152d698d7e7020f0b3173d396496"}
Nov 04 02:59:07.748820 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:07.748765    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" event={"ID":"67610923-55e2-4444-918e-ffeb7bebed07","Type":"ContainerStarted","Data":"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"}
Nov 04 02:59:07.749284 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:07.748880    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:07.866612 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:07.866553    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" podStartSLOduration=2.868006269 podStartE2EDuration="3.866539378s" totalImagesPullingTime="998.533109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:07.865352835 +0000 UTC m=+1503.006366243" watchObservedRunningTime="2026-11-04 02:59:07.866539378 +0000 UTC m=+1503.007552785"
Nov 04 02:59:10.292226 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:10.292189    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:10.739513 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:10.739426    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:11.394399 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.394367    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 02:59:11.399695 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.399671    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.454333 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.454298    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-297806-openshift-default-dockercfg-zdgvt\""
Nov 04 02:59:11.518097 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.500021    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:11.521717 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.521682    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554318    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554478    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554541    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm69k\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554589    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554747    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554779    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554885    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.554969    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.562132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.555022    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903204    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903550    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903604    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903724    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903826    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903865    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.906098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.903936    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.925103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.912792    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.925103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.913162    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.925103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.913961    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.925103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.914326    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.946095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.932540    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.946095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.932664    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm69k\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.946095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.932722    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:11.946095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.934459    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.007280 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:11.996528    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.096482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.088510    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.096482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.088934    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.096482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.090188    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm69k\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\") pod \"e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") " pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.362174 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.362145    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:12.800871 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.800843    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 02:59:12.802664 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:12.802636    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce29f7ec_27de_48f2_9479_c7e0d4712964.slice/crio-d2f4a0afdbd784ea9aa967c68cf7b4bc252c086cb8c3b87f80d96928ba4f12d3 WatchSource:0}: Error finding container d2f4a0afdbd784ea9aa967c68cf7b4bc252c086cb8c3b87f80d96928ba4f12d3: Status 404 returned error can't find the container with id d2f4a0afdbd784ea9aa967c68cf7b4bc252c086cb8c3b87f80d96928ba4f12d3
Nov 04 02:59:12.805244 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.805213    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:12.805331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.805275    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:12.805331 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.805303    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:12.905944 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.905890    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerStarted","Data":"35303b9b44f77887bbf90f43e4d154f3f410e1599ebdb623446bc0a1c055e1ad"}
Nov 04 02:59:12.906130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:12.905975    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerStarted","Data":"d2f4a0afdbd784ea9aa967c68cf7b4bc252c086cb8c3b87f80d96928ba4f12d3"}
Nov 04 02:59:13.054471 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:13.054366    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podStartSLOduration=2.054352815 podStartE2EDuration="2.054352815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:13.051302719 +0000 UTC m=+1508.192316127" watchObservedRunningTime="2026-11-04 02:59:13.054352815 +0000 UTC m=+1508.195366215"
Nov 04 02:59:13.363195 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:13.363110    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:13.364648 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:13.364616    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 02:59:13.364792 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:13.364673    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 02:59:13.761956 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:13.761900    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 02:59:14.366884 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:14.366857    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:14.924376 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:14.924346    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:14.925296 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:14.925273    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 02:59:15.136389 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.136353    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 02:59:15.140398 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.140375    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.165458 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.165408    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.166533 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.166491    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.166794 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.166712    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm6zj\" (UniqueName: \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.167164 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.167132    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.195639 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.195532    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5fde04ee-serving-cert\""
Nov 04 02:59:15.215515 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.215481    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 02:59:15.268111 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.268065    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.268268 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.268245    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.268317 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.268303    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6zj\" (UniqueName: \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.268498 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.268471    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.269103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.269024    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.269248 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:15.269206    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-serving-cert: secret "maas-api-e2e-derive-5fde04ee-serving-cert" not found
Nov 04 02:59:15.269324 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:15.269285    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls podName:1586e501-e7c0-4815-97f7-11b953664ef1 nodeName:}" failed. No retries permitted until 2026-11-04 02:59:15.769262762 +0000 UTC m=+1510.910276165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls") pod "maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" (UID: "1586e501-e7c0-4815-97f7-11b953664ef1") : secret "maas-api-e2e-derive-5fde04ee-serving-cert" not found
Nov 04 02:59:15.271117 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.271092    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.329602 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.329546    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:15.330413 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.330391    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm6zj\" (UniqueName: \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.538094 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.536584    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:15.688124 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.688046    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:15.758154 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.756322    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 02:59:15.766091 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.760111    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 02:59:15.785562 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.782459    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.787559 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.787528    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") pod \"maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:15.877763 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:15.877697    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:16.057304 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:16.057268    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:16.060658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:16.060631    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:16.228206 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:16.228174    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:16.453720 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:16.453689    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1586e501_e7c0_4815_97f7_11b953664ef1.slice/crio-f49fad3c03b7d0c9e4d84cccd89bde94c07b91870c9e7e434cdefafcbe45506f WatchSource:0}: Error finding container f49fad3c03b7d0c9e4d84cccd89bde94c07b91870c9e7e434cdefafcbe45506f: Status 404 returned error can't find the container with id f49fad3c03b7d0c9e4d84cccd89bde94c07b91870c9e7e434cdefafcbe45506f
Nov 04 02:59:16.454002 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:16.453980    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 02:59:16.938510 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:16.938476    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" event={"ID":"1586e501-e7c0-4815-97f7-11b953664ef1","Type":"ContainerStarted","Data":"f49fad3c03b7d0c9e4d84cccd89bde94c07b91870c9e7e434cdefafcbe45506f"}
Nov 04 02:59:17.946363 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:17.946325    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" event={"ID":"1586e501-e7c0-4815-97f7-11b953664ef1","Type":"ContainerStarted","Data":"f5787221159bf5279c2ed80289cd4e6004d850790ac74b5ac0b73a8b2ae6a53f"}
Nov 04 02:59:17.946771 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:17.946431    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:18.063479 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:18.063429    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" podStartSLOduration=2.185929199 podStartE2EDuration="3.063414557s" totalImagesPullingTime="877.485358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:18.062953664 +0000 UTC m=+1513.203967063" watchObservedRunningTime="2026-11-04 02:59:18.063414557 +0000 UTC m=+1513.204427963"
Nov 04 02:59:20.502240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:20.502207    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:20.887364 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:20.887329    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:21.044440 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:21.044372    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:21.226618 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:21.226535    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:21.396115 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:21.394697    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:24.020286 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:24.020251    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 02:59:25.884482 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:25.884448    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:26.071494 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:26.071460    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:26.222454 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:26.222371    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:26.439573 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:26.439542    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:29.144739 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:29.144702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:29.336240 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:29.336206    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:30.157130 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.146304    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 02:59:30.165367 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.165339    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 02:59:30.165848 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.165829    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.226350 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.226317    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xrx\" (UniqueName: \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.226502 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.226359    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.298257 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.298227    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-db4d0787\""
Nov 04 02:59:30.298403 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.298385    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-db4d0787-dockercfg-88jxn\""
Nov 04 02:59:30.328612 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.328581    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xrx\" (UniqueName: \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.328728 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.328619    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.329357 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.329340    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.389265 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.389238    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xrx\" (UniqueName: \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\") pod \"payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.521860 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.521832    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:30.901897 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:30.901858    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 02:59:30.903493 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:30.903463    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ab11be_1a02_49bb_9550_96e483d335b9.slice/crio-341a50efcc5fa392666bbee330a9c31048c60c7ed4da619052f17744af30a7b6 WatchSource:0}: Error finding container 341a50efcc5fa392666bbee330a9c31048c60c7ed4da619052f17744af30a7b6: Status 404 returned error can't find the container with id 341a50efcc5fa392666bbee330a9c31048c60c7ed4da619052f17744af30a7b6
Nov 04 02:59:31.254132 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:31.254091    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" event={"ID":"89ab11be-1a02-49bb-9550-96e483d335b9","Type":"ContainerStarted","Data":"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"}
Nov 04 02:59:31.258227 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:31.258187    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" event={"ID":"89ab11be-1a02-49bb-9550-96e483d335b9","Type":"ContainerStarted","Data":"341a50efcc5fa392666bbee330a9c31048c60c7ed4da619052f17744af30a7b6"}
Nov 04 02:59:31.267807 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:31.267763    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:31.384878 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:31.384821    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" podStartSLOduration=2.384804934 podStartE2EDuration="2.384804934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:31.383547914 +0000 UTC m=+1526.524561323" watchObservedRunningTime="2026-11-04 02:59:31.384804934 +0000 UTC m=+1526.525818351"
Nov 04 02:59:31.691414 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:31.691291    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:34.179055 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:34.179020    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:34.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:34.320909    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:36.299096 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:36.298233    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:36.795357 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:36.795319    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 02:59:37.457205 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:37.457160    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:59:37.520721 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:37.520654    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:37.716156 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:37.712639    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:59:37.964196 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:37.964153    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:59:42.715361 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:42.715324    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:43.294299 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:43.294269    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 02:59:45.475144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.475026    6019 generic.go:356] "Generic (PLEG): container finished" podID="9750f6ab-8091-494d-8228-b29f40c09080" containerID="50429ee1550b3c21b04ae167f25c6e35c96259568c1c183b7de4ccec5bfb1b74" exitCode=0
Nov 04 02:59:45.475144 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.475108    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerDied","Data":"50429ee1550b3c21b04ae167f25c6e35c96259568c1c183b7de4ccec5bfb1b74"}
Nov 04 02:59:45.475658 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.475621    6019 scope.go:122] "RemoveContainer" containerID="50429ee1550b3c21b04ae167f25c6e35c96259568c1c183b7de4ccec5bfb1b74"
Nov 04 02:59:45.477455 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.477428    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:45.477567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.477511    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:45.477567 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:45.477559    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 02:59:46.485617 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:46.485583    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerStarted","Data":"d74d08007041fa96d8965eed505c4ed2e922ca96379e08475349f9335d5b40b1"}
Nov 04 02:59:46.547056 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:46.547020    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:59:46.573263 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:46.573220    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:46.573463 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:46.573287    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:47.332086 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.332039    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 02:59:47.336641 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.336619    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.395090 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.395041    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 02:59:47.427699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.427665    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.427900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.427814    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.427983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.427906    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.428049 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.427986    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74s7\" (UniqueName: \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.440456 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.440423    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-072ece-serving-cert\""
Nov 04 02:59:47.526044 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.526005    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:47.529570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.529540    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.529714 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.529643    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.529714 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:47.529681    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-serving-cert: secret "maas-api-e2e-shared-a-w4-072ece-serving-cert" not found
Nov 04 02:59:47.529820 ip-10-0-128-201 kubenswrapper[6019]: E1104 02:59:47.529745    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls podName:fdb006a8-6c68-45e2-8eb6-ad8d5957032c nodeName:}" failed. No retries permitted until 2026-11-04 02:59:48.029721991 +0000 UTC m=+1543.170735377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls") pod "maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" (UID: "fdb006a8-6c68-45e2-8eb6-ad8d5957032c") : secret "maas-api-e2e-shared-a-w4-072ece-serving-cert" not found
Nov 04 02:59:47.529820 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.529781    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74s7\" (UniqueName: \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.529983 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.529964    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.530340 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.530312    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.532537 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.532513    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.547699 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.547670    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:47.547822 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.547746    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:47.619379 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.619270    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74s7\" (UniqueName: \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:47.727694 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.727599    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:47.881161 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:47.877323    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:48.071732 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.071698    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:48.081541 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.081506    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:48.084181 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.084156    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:48.273103 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.268367    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:48.547840 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.547761    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:48.547840 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.547812    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:48.675169 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:48.675132    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 02:59:48.678982 ip-10-0-128-201 kubenswrapper[6019]: W1104 02:59:48.678943    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb006a8_6c68_45e2_8eb6_ad8d5957032c.slice/crio-09ccb7939ac49756ceb76664dc4d78ec6624de38a720375861a22fdbeafc25c0 WatchSource:0}: Error finding container 09ccb7939ac49756ceb76664dc4d78ec6624de38a720375861a22fdbeafc25c0: Status 404 returned error can't find the container with id 09ccb7939ac49756ceb76664dc4d78ec6624de38a720375861a22fdbeafc25c0
Nov 04 02:59:49.557095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:49.552379    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:49.557095 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:49.552476    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:49.620099 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:49.619905    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" event={"ID":"fdb006a8-6c68-45e2-8eb6-ad8d5957032c","Type":"ContainerStarted","Data":"09ccb7939ac49756ceb76664dc4d78ec6624de38a720375861a22fdbeafc25c0"}
Nov 04 02:59:50.551098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.550330    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:50.551098 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.550406    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:50.733876 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.686884    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" event={"ID":"fdb006a8-6c68-45e2-8eb6-ad8d5957032c","Type":"ContainerStarted","Data":"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"}
Nov 04 02:59:50.733876 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.700752    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:50.817570 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.817496    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:59:50.832560 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:50.832511    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" podStartSLOduration=2.882683979 podStartE2EDuration="3.832495624s" totalImagesPullingTime="949.811645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:59:50.831321681 +0000 UTC m=+1545.972335088" watchObservedRunningTime="2026-11-04 02:59:50.832495624 +0000 UTC m=+1545.973509032"
Nov 04 02:59:51.547900 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:51.547868    6019 patch_prober.go:29] interesting pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:59:51.548145 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:51.547922    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:59:52.551450 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:52.551422    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:59:52.552501 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:52.552478    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 02:59:52.739146 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:52.739064    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:54.101851 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:54.101816    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 02:59:56.701799 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:56.701770    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 02:59:57.679849 ip-10-0-128-201 kubenswrapper[6019]: I1104 02:59:57.679807    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:02.585753 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:02.585722    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:06.604592 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:06.604555    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:00:07.774114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:07.774036    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:10.214259 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.214180    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"]
Nov 04 03:00:10.220144 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.220121    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.265290 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.265255    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.265542 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.265520    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjs4k\" (UniqueName: \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.265618 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.265574    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.273513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.273493    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:00:10.323338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.323302    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slbq8\""
Nov 04 03:00:10.329450 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.329423    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"]
Nov 04 03:00:10.367146 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.367113    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.367305 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.367286    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjs4k\" (UniqueName: \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.367345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.367325    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.368022 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.367992    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.369680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.369650    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.450162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.447889    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjs4k\" (UniqueName: \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\") pod \"maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:10.555387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:10.555347    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:11.119794 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:11.106679    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"]
Nov 04 03:00:11.938805 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:11.938711    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98" event={"ID":"e3afa133-d9fd-4fc5-87a4-8e093e8437c8","Type":"ContainerStarted","Data":"d0dda2c444c1119f3616d9a664817075280a856b67ebc6b1c9497e1055f6759c"}
Nov 04 03:00:11.938805 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:11.938773    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98" event={"ID":"e3afa133-d9fd-4fc5-87a4-8e093e8437c8","Type":"ContainerStarted","Data":"33c4573cd265ac25b9d8a9b793fd7f5784c4bbcd0ed41c1c962b031c130ad81c"}
Nov 04 03:00:12.057064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:12.057014    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98" podStartSLOduration=2.057000197 podStartE2EDuration="2.057000197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:00:12.053933635 +0000 UTC m=+1567.194947044" watchObservedRunningTime="2026-11-04 03:00:12.057000197 +0000 UTC m=+1567.198013604"
Nov 04 03:00:12.948358 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:12.948316    6019 generic.go:356] "Generic (PLEG): container finished" podID="e3afa133-d9fd-4fc5-87a4-8e093e8437c8" containerID="d0dda2c444c1119f3616d9a664817075280a856b67ebc6b1c9497e1055f6759c" exitCode=0
Nov 04 03:00:12.948785 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:12.948409    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98" event={"ID":"e3afa133-d9fd-4fc5-87a4-8e093e8437c8","Type":"ContainerDied","Data":"d0dda2c444c1119f3616d9a664817075280a856b67ebc6b1c9497e1055f6759c"}
Nov 04 03:00:13.077531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:13.077502    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:13.282668 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:13.282612    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:13.430116 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:13.426910    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:14.148360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.148335    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:14.269236 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.269204    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\") pod \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") "
Nov 04 03:00:14.269446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.269430    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\") pod \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") "
Nov 04 03:00:14.269513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.269483    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\" (UniqueName: \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\") pod \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\" (UID: \"e3afa133-d9fd-4fc5-87a4-8e093e8437c8\") "
Nov 04 03:00:14.269620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.269600    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca" pod "e3afa133-d9fd-4fc5-87a4-8e093e8437c8" (UID: "e3afa133-d9fd-4fc5-87a4-8e093e8437c8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:00:14.270493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.270469    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-openshift-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:14.271976 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.271949    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k" pod "e3afa133-d9fd-4fc5-87a4-8e093e8437c8" (UID: "e3afa133-d9fd-4fc5-87a4-8e093e8437c8"). InnerVolumeSpecName "kube-api-access-sjs4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:00:14.272596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.272567    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca" pod "e3afa133-d9fd-4fc5-87a4-8e093e8437c8" (UID: "e3afa133-d9fd-4fc5-87a4-8e093e8437c8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:00:14.372030 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.371987    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-maas-api-xks-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:14.372030 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.372025    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjs4k\" (UniqueName: \"kubernetes.io/projected/e3afa133-d9fd-4fc5-87a4-8e093e8437c8-kube-api-access-sjs4k\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:14.971982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.971952    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"
Nov 04 03:00:14.972236 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.971983    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98" event={"ID":"e3afa133-d9fd-4fc5-87a4-8e093e8437c8","Type":"ContainerDied","Data":"33c4573cd265ac25b9d8a9b793fd7f5784c4bbcd0ed41c1c962b031c130ad81c"}
Nov 04 03:00:14.972236 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:14.972060    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33c4573cd265ac25b9d8a9b793fd7f5784c4bbcd0ed41c1c962b031c130ad81c"
Nov 04 03:00:18.508691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:18.508656    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:21.238001 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:21.237910    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:21.392145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:21.392062    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:22.227301 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:22.227258    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:22.371276 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:22.371241    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:23.893211 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:23.893176    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:25.033146 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:25.033120    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:28.760614 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.760587    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:00:28.761311 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.761296    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e3afa133-d9fd-4fc5-87a4-8e093e8437c8" containerName="revoke-keys"
Nov 04 03:00:28.761369 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.761312    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3afa133-d9fd-4fc5-87a4-8e093e8437c8" containerName="revoke-keys"
Nov 04 03:00:28.761468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.761460    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3afa133-d9fd-4fc5-87a4-8e093e8437c8" containerName="revoke-keys"
Nov 04 03:00:28.765220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.765203    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.817509 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.817485    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-db4d0787-serving-cert\""
Nov 04 03:00:28.869972 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.869940    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.870195 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.870172    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.870270 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.870243    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-554jb\" (UniqueName: \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.870417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.870394    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.870482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.870434    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:00:28.958401 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.958372    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:28.972240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.972210    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.972435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.972304    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-554jb\" (UniqueName: \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.972435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.972431    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.972566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.972550    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.972708 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:28.972680    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-serving-cert: secret "maas-api-e2e-mt-db4d0787-serving-cert" not found
Nov 04 03:00:28.972823 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:28.972750    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls podName:7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381 nodeName:}" failed. No retries permitted until 2026-11-04 03:00:29.472735411 +0000 UTC m=+1584.613748795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls") pod "maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" (UID: "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381") : secret "maas-api-e2e-mt-db4d0787-serving-cert" not found
Nov 04 03:00:28.973027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.973009    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:28.974889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:28.974871    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:29.094144 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.094000    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-554jb\" (UniqueName: \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:29.120930 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.120891    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:29.297095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.297035    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:29.494012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.493915    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:29.496455 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.496432    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") pod \"maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:29.688026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:29.687994    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:30.071308 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:30.071281    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:00:30.072656 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:00:30.072631    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa4cd2a_91fc_4c77_ba34_e58ad3ba4381.slice/crio-23702d8aa8ebdaaf8c5efc995aaf87cb175a347d8537a55a973ab9397406d130 WatchSource:0}: Error finding container 23702d8aa8ebdaaf8c5efc995aaf87cb175a347d8537a55a973ab9397406d130: Status 404 returned error can't find the container with id 23702d8aa8ebdaaf8c5efc995aaf87cb175a347d8537a55a973ab9397406d130
Nov 04 03:00:30.195179 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:30.195147    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" event={"ID":"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381","Type":"ContainerStarted","Data":"23702d8aa8ebdaaf8c5efc995aaf87cb175a347d8537a55a973ab9397406d130"}
Nov 04 03:00:31.204463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:31.204373    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" event={"ID":"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381","Type":"ContainerStarted","Data":"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"}
Nov 04 03:00:31.204896 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:31.204532    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:31.325149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:31.325085    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" podStartSLOduration=2.5432521660000003 podStartE2EDuration="3.325053066s" totalImagesPullingTime="781.8009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:00:31.321612382 +0000 UTC m=+1586.462625792" watchObservedRunningTime="2026-11-04 03:00:31.325053066 +0000 UTC m=+1586.466066510"
Nov 04 03:00:34.178766 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:34.178729    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:34.485888 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:34.485792    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:36.606221 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:36.606184    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:00:36.654041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:36.654011    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:37.235688 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:37.235663    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:00:38.574277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:38.574242    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:38.759282 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:38.759246    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:39.102513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:39.102477    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:43.984956 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:43.984920    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:48.475291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:48.475253    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:48.862978 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:48.862942    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:52.041663 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:52.041586    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 03:00:52.043120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:52.043031    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" containerID="cri-o://880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64" gracePeriod=30
Nov 04 03:00:52.083441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:52.083399    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 03:00:52.083706 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:52.083680    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" containerID="cri-o://8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba" gracePeriod=30
Nov 04 03:00:53.696672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:53.696639    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:53.849217 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:53.849171    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:00:53.849504 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:53.849469    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:00:54.513671 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:54.513640    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:56.315026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:56.314992    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:00:56.315455 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:56.315050    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:00:57.429953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.429918    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 03:00:57.433575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.433556    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 03:00:57.542179 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542061    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542252    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.542378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542292    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.542378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542334    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.542548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542371    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.542548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542383    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.542548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542473    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542513    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542543    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542665    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542753    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542930 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542843    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.542930 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542875    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.542930 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542881    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.542930 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542926    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.543171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.542963    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.543171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543047    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.543171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543091    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.543171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543131    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.543550 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543523    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.543645 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543622    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.543838 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543732    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\") pod \"29f0cb07-c8c9-4b50-a467-c328951ace6c\" (UID: \"29f0cb07-c8c9-4b50-a467-c328951ace6c\") "
Nov 04 03:00:57.544952 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.544539    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\") pod \"97ac5484-4613-4022-984a-d34327ae5661\" (UID: \"97ac5484-4613-4022-984a-d34327ae5661\") "
Nov 04 03:00:57.545089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.543783    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.545089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.544148    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.545089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.544345    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.545089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.544465    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:00:57.545089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.544912    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:00:57.545939 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.545914    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.545944    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.545960    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ac5484-4613-4022-984a-d34327ae5661-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.545974    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.545987    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.546003    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.546047    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29f0cb07-c8c9-4b50-a467-c328951ace6c-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.546326    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.546342    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.546433 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.546356    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.548641 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.548616    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.548751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.548643    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:00:57.548751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.548656    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:00:57.548968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.548881    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:00:57.549101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.549047    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:00:57.549176 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.549104    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "kube-api-access-9smhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:00:57.549382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.549360    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo" pod "97ac5484-4613-4022-984a-d34327ae5661" (UID: "97ac5484-4613-4022-984a-d34327ae5661"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:00:57.550293 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.550274    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq" pod "29f0cb07-c8c9-4b50-a467-c328951ace6c" (UID: "29f0cb07-c8c9-4b50-a467-c328951ace6c"). InnerVolumeSpecName "kube-api-access-hbzlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:00:57.635754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.635670    6019 generic.go:356] "Generic (PLEG): container finished" podID="97ac5484-4613-4022-984a-d34327ae5661" containerID="880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64" exitCode=0
Nov 04 03:00:57.635935 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.635776    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"
Nov 04 03:00:57.635935 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.635812    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" event={"ID":"97ac5484-4613-4022-984a-d34327ae5661","Type":"ContainerDied","Data":"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"}
Nov 04 03:00:57.635935 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.635914    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl" event={"ID":"97ac5484-4613-4022-984a-d34327ae5661","Type":"ContainerDied","Data":"d85bdabd13e05109c47ed4db0840d1e45c3af0d8e40da6a8c823ef6efeee514f"}
Nov 04 03:00:57.636186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.635946    6019 scope.go:122] "RemoveContainer" containerID="880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"
Nov 04 03:00:57.637777 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.637755    6019 generic.go:356] "Generic (PLEG): container finished" podID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerID="8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba" exitCode=0
Nov 04 03:00:57.637892 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.637810    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" event={"ID":"29f0cb07-c8c9-4b50-a467-c328951ace6c","Type":"ContainerDied","Data":"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"}
Nov 04 03:00:57.637892 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.637849    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn" event={"ID":"29f0cb07-c8c9-4b50-a467-c328951ace6c","Type":"ContainerDied","Data":"12b89d2de9b0204ac422bcd5aacb29a6798f5627ec2d306b33fa2b9554398ca8"}
Nov 04 03:00:57.638115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.637929    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"
Nov 04 03:00:57.647780 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647761    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ac5484-4613-4022-984a-d34327ae5661-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647783    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647793    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ac5484-4613-4022-984a-d34327ae5661-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647801    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9smhc\" (UniqueName: \"kubernetes.io/projected/97ac5484-4613-4022-984a-d34327ae5661-kube-api-access-9smhc\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647812    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647820    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647829    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbzlq\" (UniqueName: \"kubernetes.io/projected/29f0cb07-c8c9-4b50-a467-c328951ace6c-kube-api-access-hbzlq\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.647868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.647837    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29f0cb07-c8c9-4b50-a467-c328951ace6c-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:00:57.651860 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.651839    6019 scope.go:122] "RemoveContainer" containerID="880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"
Nov 04 03:00:57.652243 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:57.652216    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64\": container with ID starting with 880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64 not found: ID does not exist" containerID="880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"
Nov 04 03:00:57.652323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.652255    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64"} err="failed to get container status \"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64\": rpc error: code = NotFound desc = could not find container \"880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64\": container with ID starting with 880c7f74a65aaff9e787fd2af25d1a8e021d75c273d1e18dfa75838218496b64 not found: ID does not exist"
Nov 04 03:00:57.652323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.652278    6019 scope.go:122] "RemoveContainer" containerID="8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"
Nov 04 03:00:57.662496 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.662476    6019 scope.go:122] "RemoveContainer" containerID="8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"
Nov 04 03:00:57.662802 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:00:57.662761    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba\": container with ID starting with 8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba not found: ID does not exist" containerID="8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"
Nov 04 03:00:57.662872 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.662813    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba"} err="failed to get container status \"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba\": rpc error: code = NotFound desc = could not find container \"8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba\": container with ID starting with 8e69f6b62adfd118392e756a1e08d054f2b45dc5fd32b1a85d1f9501848c00ba not found: ID does not exist"
Nov 04 03:00:57.834304 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.834258    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 03:00:57.837819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.837791    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn"]
Nov 04 03:00:57.995173 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:57.994114    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 03:00:58.010954 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:58.008596    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl"]
Nov 04 03:00:58.820345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:58.820313    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:00:59.008128 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:59.008093    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" path="/var/lib/kubelet/pods/29f0cb07-c8c9-4b50-a467-c328951ace6c/volumes"
Nov 04 03:00:59.008787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:00:59.008756    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97ac5484-4613-4022-984a-d34327ae5661" path="/var/lib/kubelet/pods/97ac5484-4613-4022-984a-d34327ae5661/volumes"
Nov 04 03:01:04.122441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:04.122372    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:04.277505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:04.277447    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:04.626576 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:04.626537    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:05.536556 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:05.536519    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:05.759095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:05.758655    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:06.605668 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:01:06.605628    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:01:08.652631 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:08.652599    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 03:01:08.653203 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:08.652973    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" podUID="67610923-55e2-4444-918e-ffeb7bebed07" containerName="maas-api" containerID="cri-o://9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f" gracePeriod=30
Nov 04 03:01:08.753799 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:08.753757    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:08.756867 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:08.756840    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" podUID="67610923-55e2-4444-918e-ffeb7bebed07" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.86:8443/health\": dial tcp 10.132.0.86:8443: connect: connection refused"
Nov 04 03:01:09.240104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.235432    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 03:01:09.363817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.361656    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") pod \"67610923-55e2-4444-918e-ffeb7bebed07\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") "
Nov 04 03:01:09.363817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.361747    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\") pod \"67610923-55e2-4444-918e-ffeb7bebed07\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") "
Nov 04 03:01:09.363817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.361787    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\") pod \"67610923-55e2-4444-918e-ffeb7bebed07\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") "
Nov 04 03:01:09.363817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.361966    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\" (UniqueName: \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\") pod \"67610923-55e2-4444-918e-ffeb7bebed07\" (UID: \"67610923-55e2-4444-918e-ffeb7bebed07\") "
Nov 04 03:01:09.369613 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.369569    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca" pod "67610923-55e2-4444-918e-ffeb7bebed07" (UID: "67610923-55e2-4444-918e-ffeb7bebed07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:01:09.408596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.376122    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67610923-55e2-4444-918e-ffeb7bebed07-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:09.421572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.408574    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs" pod "67610923-55e2-4444-918e-ffeb7bebed07" (UID: "67610923-55e2-4444-918e-ffeb7bebed07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:01:09.421572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.411939    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls" pod "67610923-55e2-4444-918e-ffeb7bebed07" (UID: "67610923-55e2-4444-918e-ffeb7bebed07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:01:09.422516 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.422460    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg" pod "67610923-55e2-4444-918e-ffeb7bebed07" (UID: "67610923-55e2-4444-918e-ffeb7bebed07"). InnerVolumeSpecName "kube-api-access-mb2rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:09.551251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.525790    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:09.551251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.525827    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67610923-55e2-4444-918e-ffeb7bebed07-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:09.551251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.525845    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb2rg\" (UniqueName: \"kubernetes.io/projected/67610923-55e2-4444-918e-ffeb7bebed07-kube-api-access-mb2rg\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:09.806920 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.806642    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" event={"ID":"67610923-55e2-4444-918e-ffeb7bebed07","Type":"ContainerDied","Data":"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"}
Nov 04 03:01:09.806920 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.806765    6019 scope.go:122] "RemoveContainer" containerID="9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"
Nov 04 03:01:09.809636 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.806424    6019 generic.go:356] "Generic (PLEG): container finished" podID="67610923-55e2-4444-918e-ffeb7bebed07" containerID="9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f" exitCode=0
Nov 04 03:01:09.809636 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.809579    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl" event={"ID":"67610923-55e2-4444-918e-ffeb7bebed07","Type":"ContainerDied","Data":"58fc2f233418a98f400d2bcad371a4bd17cf152d698d7e7020f0b3173d396496"}
Nov 04 03:01:09.819153 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.818926    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"
Nov 04 03:01:09.962453 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.962421    6019 scope.go:122] "RemoveContainer" containerID="9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"
Nov 04 03:01:09.969278 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:01:09.969226    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f\": container with ID starting with 9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f not found: ID does not exist" containerID="9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"
Nov 04 03:01:09.969519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:09.969479    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f"} err="failed to get container status \"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f\": rpc error: code = NotFound desc = could not find container \"9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f\": container with ID starting with 9455c2bc40f40c5a701395d40626ad8ef0f6f826e79d57dbad47eb3426946c8f not found: ID does not exist"
Nov 04 03:01:10.022268 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.022231    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:10.035208 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.035177    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 03:01:10.038125 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.038090    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4d5a0200-66cc4bcf75-jt2pl"]
Nov 04 03:01:10.236934 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.236854    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:10.381711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.381674    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:10.721843 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.721810    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:10.889058 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.887761    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:10.987016 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:10.986942    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67610923-55e2-4444-918e-ffeb7bebed07" path="/var/lib/kubelet/pods/67610923-55e2-4444-918e-ffeb7bebed07/volumes"
Nov 04 03:01:11.163671 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:11.163637    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:11.389366 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:11.389332    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"]
Nov 04 03:01:11.438937 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:11.438902    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4d5a0200-q9c98"]
Nov 04 03:01:12.887377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:12.887343    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 03:01:12.887871 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:12.887597    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy" containerID="cri-o://d74d08007041fa96d8965eed505c4ed2e922ca96379e08475349f9335d5b40b1" gracePeriod=30
Nov 04 03:01:12.898362 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:12.898336    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:13.083109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:13.081745    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3afa133-d9fd-4fc5-87a4-8e093e8437c8" path="/var/lib/kubelet/pods/e3afa133-d9fd-4fc5-87a4-8e093e8437c8/volumes"
Nov 04 03:01:13.507025 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:13.506990    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:13.859591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:13.859485    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:14.014993 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:14.014953    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:14.629158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:14.623670    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:14.890634 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:14.890545    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:16.366153 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:16.366084    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:18.091951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.091706    6019 generic.go:356] "Generic (PLEG): container finished" podID="9750f6ab-8091-494d-8228-b29f40c09080" containerID="d74d08007041fa96d8965eed505c4ed2e922ca96379e08475349f9335d5b40b1" exitCode=0
Nov 04 03:01:18.091951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.091738    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerDied","Data":"d74d08007041fa96d8965eed505c4ed2e922ca96379e08475349f9335d5b40b1"}
Nov 04 03:01:18.091951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.091822    6019 scope.go:122] "RemoveContainer" containerID="50429ee1550b3c21b04ae167f25c6e35c96259568c1c183b7de4ccec5bfb1b74"
Nov 04 03:01:18.181942 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.181921    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 03:01:18.272596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272563    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.272801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272613    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.272801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272680    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.272801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272712    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.272801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272749    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.273012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.272852    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.273174 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273136    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.273290 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273209    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.273290 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273224    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:18.273410 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273299    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:18.273410 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273354    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:18.273592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273498    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:18.273681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.273650    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\") pod \"9750f6ab-8091-494d-8228-b29f40c09080\" (UID: \"9750f6ab-8091-494d-8228-b29f40c09080\") "
Nov 04 03:01:18.274266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.274235    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:01:18.274417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.274312    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.274417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.274333    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.274417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.274348    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.274417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.274361    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.275628 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.275598    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:01:18.275770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.275752    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "kube-api-access-w8lnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:18.275840 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.275829    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:18.276478 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.276459    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy" pod "9750f6ab-8091-494d-8228-b29f40c09080" (UID: "9750f6ab-8091-494d-8228-b29f40c09080"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:18.376650 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.376610    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8lnq\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-kube-api-access-w8lnq\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.376650 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.376653    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9750f6ab-8091-494d-8228-b29f40c09080-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.376899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.376670    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9750f6ab-8091-494d-8228-b29f40c09080-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.376899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.376684    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9750f6ab-8091-494d-8228-b29f40c09080-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.376899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.376697    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9750f6ab-8091-494d-8228-b29f40c09080-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:18.942273 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:18.942222    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:19.099866 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.099831    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9" event={"ID":"9750f6ab-8091-494d-8228-b29f40c09080","Type":"ContainerDied","Data":"8e0e17897be845c517d63e379488435ce6ded0cb349e00697f787af77790a54f"}
Nov 04 03:01:19.099866 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.099854    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"
Nov 04 03:01:19.100414 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.099891    6019 scope.go:122] "RemoveContainer" containerID="d74d08007041fa96d8965eed505c4ed2e922ca96379e08475349f9335d5b40b1"
Nov 04 03:01:19.189348 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.189321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:19.317567 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.317530    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 03:01:19.342971 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.342942    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:19.364501 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:19.364473    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9"]
Nov 04 03:01:20.187137 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:20.184425    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:20.363253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:20.363128    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:20.819094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:20.816957    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 03:01:20.851099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:20.831409    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" podUID="1586e501-e7c0-4815-97f7-11b953664ef1" containerName="maas-api" containerID="cri-o://f5787221159bf5279c2ed80289cd4e6004d850790ac74b5ac0b73a8b2ae6a53f" gracePeriod=30
Nov 04 03:01:20.967036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:20.967001    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:21.171159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.166478    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9750f6ab-8091-494d-8228-b29f40c09080" path="/var/lib/kubelet/pods/9750f6ab-8091-494d-8228-b29f40c09080/volumes"
Nov 04 03:01:21.295274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.295244    6019 generic.go:356] "Generic (PLEG): container finished" podID="1586e501-e7c0-4815-97f7-11b953664ef1" containerID="f5787221159bf5279c2ed80289cd4e6004d850790ac74b5ac0b73a8b2ae6a53f" exitCode=0
Nov 04 03:01:21.302309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.302274    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" event={"ID":"1586e501-e7c0-4815-97f7-11b953664ef1","Type":"ContainerDied","Data":"f5787221159bf5279c2ed80289cd4e6004d850790ac74b5ac0b73a8b2ae6a53f"}
Nov 04 03:01:21.871131 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.868655    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 03:01:21.922784 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.922725    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\" (UniqueName: \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\") pod \"1586e501-e7c0-4815-97f7-11b953664ef1\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") "
Nov 04 03:01:21.922999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.922863    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\") pod \"1586e501-e7c0-4815-97f7-11b953664ef1\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") "
Nov 04 03:01:21.922999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.922910    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") pod \"1586e501-e7c0-4815-97f7-11b953664ef1\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") "
Nov 04 03:01:21.923155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.923042    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\") pod \"1586e501-e7c0-4815-97f7-11b953664ef1\" (UID: \"1586e501-e7c0-4815-97f7-11b953664ef1\") "
Nov 04 03:01:21.927157 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.926436    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca" pod "1586e501-e7c0-4815-97f7-11b953664ef1" (UID: "1586e501-e7c0-4815-97f7-11b953664ef1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:01:21.934172 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.931311    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1586e501-e7c0-4815-97f7-11b953664ef1-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:21.934671 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.934618    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj" pod "1586e501-e7c0-4815-97f7-11b953664ef1" (UID: "1586e501-e7c0-4815-97f7-11b953664ef1"). InnerVolumeSpecName "kube-api-access-xm6zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:21.947108 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.944198    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls" pod "1586e501-e7c0-4815-97f7-11b953664ef1" (UID: "1586e501-e7c0-4815-97f7-11b953664ef1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:01:21.955171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:21.951232    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs" pod "1586e501-e7c0-4815-97f7-11b953664ef1" (UID: "1586e501-e7c0-4815-97f7-11b953664ef1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:01:22.042341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.042197    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm6zj\" (UniqueName: \"kubernetes.io/projected/1586e501-e7c0-4815-97f7-11b953664ef1-kube-api-access-xm6zj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:22.042341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.042234    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:22.042341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.042248    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1586e501-e7c0-4815-97f7-11b953664ef1-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:22.409103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.398784    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m" event={"ID":"1586e501-e7c0-4815-97f7-11b953664ef1","Type":"ContainerDied","Data":"f49fad3c03b7d0c9e4d84cccd89bde94c07b91870c9e7e434cdefafcbe45506f"}
Nov 04 03:01:22.409103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.398895    6019 scope.go:122] "RemoveContainer" containerID="f5787221159bf5279c2ed80289cd4e6004d850790ac74b5ac0b73a8b2ae6a53f"
Nov 04 03:01:22.409103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.399115    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"
Nov 04 03:01:22.619302 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.619273    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 03:01:22.620588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.620569    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fde04ee-6fbc78f94d-p978m"]
Nov 04 03:01:22.990513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:22.990474    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1586e501-e7c0-4815-97f7-11b953664ef1" path="/var/lib/kubelet/pods/1586e501-e7c0-4815-97f7-11b953664ef1/volumes"
Nov 04 03:01:23.113446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:23.113415    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:23.773365 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:23.773190    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:23.918577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:23.918541    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:24.138717 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:24.138636    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:24.287113 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:24.283620    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:24.978748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:24.978712    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:25.935223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:25.935194    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.086451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.086398    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.296896 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.296859    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.497868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.497825    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.644940 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.644786    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.796661 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.796547    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:26.987147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:26.986980    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:27.386098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:27.385252    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:27.637157 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:27.636444    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:27.787158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:27.787122    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:28.730541 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:28.730500    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:30.966201 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:30.966169    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:01:30.966605 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:30.966227    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:01:31.792390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:31.792352    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:31.961629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:31.961558    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:32.107188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.107111    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:32.605631 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.605602    6019 generic.go:356] "Generic (PLEG): container finished" podID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerID="35303b9b44f77887bbf90f43e4d154f3f410e1599ebdb623446bc0a1c055e1ad" exitCode=0
Nov 04 03:01:32.605804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.605646    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerDied","Data":"35303b9b44f77887bbf90f43e4d154f3f410e1599ebdb623446bc0a1c055e1ad"}
Nov 04 03:01:32.606085 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.606052    6019 scope.go:122] "RemoveContainer" containerID="35303b9b44f77887bbf90f43e4d154f3f410e1599ebdb623446bc0a1c055e1ad"
Nov 04 03:01:32.607716 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.607681    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:01:32.607839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.607752    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:01:32.607839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:32.607784    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:01:33.619242 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:33.619204    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerStarted","Data":"c98653cf7e82af7e30259e2b52363dd59d44dc914d35931917d76e1664682a2d"}
Nov 04 03:01:33.982990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:33.982900    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:34.219256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.209369    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:34.393111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.366730    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:34.393111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.368979    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:01:34.393111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.369195    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 03:01:34.393111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.369279    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 03:01:34.933608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:34.933527    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:35.369763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:35.369729    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 03:01:35.370058 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:35.370024    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 03:01:36.366509 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:36.366471    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 03:01:36.367151 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:36.367119    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 03:01:36.673383 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:01:36.671696    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:01:37.009487 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:37.009435    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:37.364586 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:37.364487    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 03:01:37.364877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:37.364848    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 03:01:38.369606 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:38.369567    6019 patch_prober.go:29] interesting pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 03:01:38.370263 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:38.370233    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 03:01:38.994291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:38.994241    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:39.499251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:39.498982    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:01:39.500037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:39.500017    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:01:39.501402 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:39.501351    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:01:39.794709 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:39.794614    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:39.943580 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:39.943543    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:40.608291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:40.608261    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:41.564244 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:41.564208    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:41.775135 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:41.767058    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:45.037104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:45.034662    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:45.440593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:45.440505    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:45.587314 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:45.587278    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.027100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.002663    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.224103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.222793    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.558163 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.558122    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.722093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.721847    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.854152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.852208    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:01:47.854152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.852289    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:01:47.883094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.880941    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:47.891105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.887176    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:01:47.891105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:47.887232    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:01:49.996225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:49.996185    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:51.692789 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:51.692758    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:52.929647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:52.929607    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 03:01:52.930103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:52.929813    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy" containerID="cri-o://4a649331bf7e830ac3b05fa7ed2817a9e48d26ad971bfe14330c2b77886bb8b7" gracePeriod=30
Nov 04 03:01:53.840426 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:53.840384    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:54.153926 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:54.153837    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 03:01:54.155037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:54.154064    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" containerID="cri-o://7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb" gracePeriod=30
Nov 04 03:01:55.036338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:55.036290    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:56.181960 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:56.181923    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 03:01:56.426647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:56.426612    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:56.587815 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:56.587779    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:01:58.124002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.123972    6019 generic.go:356] "Generic (PLEG): container finished" podID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerID="4a649331bf7e830ac3b05fa7ed2817a9e48d26ad971bfe14330c2b77886bb8b7" exitCode=0
Nov 04 03:01:58.124485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.124031    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" event={"ID":"aabe9ac3-8823-4883-b0f3-fe9b19562187","Type":"ContainerDied","Data":"4a649331bf7e830ac3b05fa7ed2817a9e48d26ad971bfe14330c2b77886bb8b7"}
Nov 04 03:01:58.212277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.212217    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 03:01:58.304651 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304622    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.304839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304672    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.304839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304732    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.304839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304786    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.305011 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304913    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.305011 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304956    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.305011 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.304987    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.305190 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305048    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:58.305277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305256    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.305343 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305289    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:58.305343 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305310    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:58.305457 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305411    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:01:58.305544 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305509    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:58.305617 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.305581    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\") pod \"aabe9ac3-8823-4883-b0f3-fe9b19562187\" (UID: \"aabe9ac3-8823-4883-b0f3-fe9b19562187\") "
Nov 04 03:01:58.306053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.306032    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.306053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.306056    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabe9ac3-8823-4883-b0f3-fe9b19562187-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.306441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.306082    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.306441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.306093    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.306441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.306101    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.307168 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.307143    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:58.307607 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.307581    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:01:58.307694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.307607    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:58.307965 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.307947    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2" pod "aabe9ac3-8823-4883-b0f3-fe9b19562187" (UID: "aabe9ac3-8823-4883-b0f3-fe9b19562187"). InnerVolumeSpecName "kube-api-access-nsbm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:58.407708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.407672    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsbm2\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-kube-api-access-nsbm2\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.407708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.407708    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.407966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.407724    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:58.407966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:58.407737    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabe9ac3-8823-4883-b0f3-fe9b19562187-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.133271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.133232    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"
Nov 04 03:01:59.133271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.133256    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r" event={"ID":"aabe9ac3-8823-4883-b0f3-fe9b19562187","Type":"ContainerDied","Data":"2367218138b6cf8167f0155c260e84e619ec5176cd39510c91531cb9f2d2e72a"}
Nov 04 03:01:59.133832 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.133336    6019 scope.go:122] "RemoveContainer" containerID="4a649331bf7e830ac3b05fa7ed2817a9e48d26ad971bfe14330c2b77886bb8b7"
Nov 04 03:01:59.313425 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.313390    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 03:01:59.313586 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.313453    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 03:01:59.349976 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.349944    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 03:01:59.399232 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.399205    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r"]
Nov 04 03:01:59.426857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.426835    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 03:01:59.524894 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.524822    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.524894 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.524873    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.524894 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.524896    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525209 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.524930    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525209 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.524970    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525209 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525027    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525232    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:01:59.525377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525267    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525314    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525541 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525373    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:59.525541 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525416    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\") pod \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\" (UID: \"1f68d9f7-8ed0-4687-b7a4-97745a8d0197\") "
Nov 04 03:01:59.525541 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525511    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:59.525755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525735    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:59.525920 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.525791    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:59.526466 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.526446    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.526515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.526476    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.526515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.526492    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.526515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.526506    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.526609 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.526520    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.527378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.527349    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "kube-api-access-wp9vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:59.527474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.527415    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:01:59.527913 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.527897    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:01:59.527952 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.527920    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy" pod "1f68d9f7-8ed0-4687-b7a4-97745a8d0197" (UID: "1f68d9f7-8ed0-4687-b7a4-97745a8d0197"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:01:59.627746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.627709    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.627746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.627742    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp9vz\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-kube-api-access-wp9vz\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.627746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.627753    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.627971 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.627762    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f68d9f7-8ed0-4687-b7a4-97745a8d0197-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:01:59.919140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:01:59.919013    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:00.141968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.141933    6019 generic.go:356] "Generic (PLEG): container finished" podID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerID="7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb" exitCode=0
Nov 04 03:02:00.142492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.141992    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"
Nov 04 03:02:00.142492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.142014    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" event={"ID":"1f68d9f7-8ed0-4687-b7a4-97745a8d0197","Type":"ContainerDied","Data":"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"}
Nov 04 03:02:00.142492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.142102    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s" event={"ID":"1f68d9f7-8ed0-4687-b7a4-97745a8d0197","Type":"ContainerDied","Data":"b8b9a09d7dd8aaf3c0777eb53d426e63adfbce0f396f6618bb55b654bbcea43c"}
Nov 04 03:02:00.142492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.142141    6019 scope.go:122] "RemoveContainer" containerID="7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"
Nov 04 03:02:00.154817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.154797    6019 scope.go:122] "RemoveContainer" containerID="7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"
Nov 04 03:02:00.155141 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:00.155117    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb\": container with ID starting with 7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb not found: ID does not exist" containerID="7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"
Nov 04 03:02:00.155197 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.155151    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb"} err="failed to get container status \"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb\": rpc error: code = NotFound desc = could not find container \"7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb\": container with ID starting with 7190aff4267d45c3160e398df0edce02e15e928075a7b3178b3a822dcdcbb8fb not found: ID does not exist"
Nov 04 03:02:00.296792 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.296764    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:00.307979 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.307953    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 03:02:00.312205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.312184    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s"]
Nov 04 03:02:00.983732 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.983698    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" path="/var/lib/kubelet/pods/1f68d9f7-8ed0-4687-b7a4-97745a8d0197/volumes"
Nov 04 03:02:00.984274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:00.984254    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" path="/var/lib/kubelet/pods/aabe9ac3-8823-4883-b0f3-fe9b19562187/volumes"
Nov 04 03:02:01.183175 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.183126    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" podUID="89ab11be-1a02-49bb-9550-96e483d335b9" containerName="payload-pre-processing" containerID="cri-o://75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982" gracePeriod=25
Nov 04 03:02:01.454054 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.454023    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 03:02:01.559320 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.559243    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\" (UniqueName: \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\") pod \"89ab11be-1a02-49bb-9550-96e483d335b9\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") "
Nov 04 03:02:01.559471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.559432    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\") pod \"89ab11be-1a02-49bb-9550-96e483d335b9\" (UID: \"89ab11be-1a02-49bb-9550-96e483d335b9\") "
Nov 04 03:02:01.559858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.559835    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume" pod "89ab11be-1a02-49bb-9550-96e483d335b9" (UID: "89ab11be-1a02-49bb-9550-96e483d335b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:02:01.559942 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.559922    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89ab11be-1a02-49bb-9550-96e483d335b9-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:01.561533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.561508    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx" pod "89ab11be-1a02-49bb-9550-96e483d335b9" (UID: "89ab11be-1a02-49bb-9550-96e483d335b9"). InnerVolumeSpecName "kube-api-access-z5xrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:02:01.661445 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:01.661411    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5xrx\" (UniqueName: \"kubernetes.io/projected/89ab11be-1a02-49bb-9550-96e483d335b9-kube-api-access-z5xrx\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:02.161530 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.161499    6019 generic.go:356] "Generic (PLEG): container finished" podID="89ab11be-1a02-49bb-9550-96e483d335b9" containerID="75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982" exitCode=0
Nov 04 03:02:02.161707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.161547    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"
Nov 04 03:02:02.161707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.161591    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" event={"ID":"89ab11be-1a02-49bb-9550-96e483d335b9","Type":"ContainerDied","Data":"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"}
Nov 04 03:02:02.161707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.161640    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5" event={"ID":"89ab11be-1a02-49bb-9550-96e483d335b9","Type":"ContainerDied","Data":"341a50efcc5fa392666bbee330a9c31048c60c7ed4da619052f17744af30a7b6"}
Nov 04 03:02:02.161707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.161661    6019 scope.go:122] "RemoveContainer" containerID="75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"
Nov 04 03:02:02.172646 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.172510    6019 scope.go:122] "RemoveContainer" containerID="75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"
Nov 04 03:02:02.172836 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:02.172813    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982\": container with ID starting with 75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982 not found: ID does not exist" containerID="75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"
Nov 04 03:02:02.172888 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.172848    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982"} err="failed to get container status \"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982\": rpc error: code = NotFound desc = could not find container \"75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982\": container with ID starting with 75b3222e28ee20165b384069874bdf4ccf860a246e3662eea46a3282d4da0982 not found: ID does not exist"
Nov 04 03:02:02.327695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.327664    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 03:02:02.377221 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.377189    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5"]
Nov 04 03:02:02.983531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:02.983498    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89ab11be-1a02-49bb-9550-96e483d335b9" path="/var/lib/kubelet/pods/89ab11be-1a02-49bb-9550-96e483d335b9/volumes"
Nov 04 03:02:05.167216 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:05.167182    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:06.608254 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:06.608217    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:02:09.929119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:09.929052    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:10.101096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:10.100784    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:10.256154 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:10.247607    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:15.018495 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:15.018457    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:19.332270 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:19.332227    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:19.964777 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:19.964643    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:20.221955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:20.221848    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:20.471676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:20.471640    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:24.629535 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:24.629498    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:24.828189 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:24.828143    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:24.979622 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:24.979528    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:25.391459 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:25.391426    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:25.552167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:25.552100    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:25.784620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:25.784582    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:30.491266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.491236    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:30.558973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.558939    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:02:30.559599 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559585    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1586e501-e7c0-4815-97f7-11b953664ef1" containerName="maas-api"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559600    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1586e501-e7c0-4815-97f7-11b953664ef1" containerName="maas-api"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559620    6019 cpu_manager.go:414] "containerMap: removing container" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559625    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559632    6019 cpu_manager.go:414] "containerMap: removing container" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559637    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559646    6019 cpu_manager.go:414] "containerMap: removing container" podUID="67610923-55e2-4444-918e-ffeb7bebed07" containerName="maas-api"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559650    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67610923-55e2-4444-918e-ffeb7bebed07" containerName="maas-api"
Nov 04 03:02:30.559664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559664    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559669    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559679    6019 cpu_manager.go:414] "containerMap: removing container" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559683    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559698    6019 cpu_manager.go:414] "containerMap: removing container" podUID="89ab11be-1a02-49bb-9550-96e483d335b9" containerName="payload-pre-processing"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559703    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89ab11be-1a02-49bb-9550-96e483d335b9" containerName="payload-pre-processing"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559713    6019 cpu_manager.go:414] "containerMap: removing container" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559718    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559732    6019 cpu_manager.go:414] "containerMap: removing container" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559736    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559839    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="29f0cb07-c8c9-4b50-a467-c328951ace6c" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559848    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559855    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="67610923-55e2-4444-918e-ffeb7bebed07" containerName="maas-api"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559862    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f68d9f7-8ed0-4687-b7a4-97745a8d0197" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559869    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="9750f6ab-8091-494d-8228-b29f40c09080" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559877    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="89ab11be-1a02-49bb-9550-96e483d335b9" containerName="payload-pre-processing"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559884    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="aabe9ac3-8823-4883-b0f3-fe9b19562187" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559891    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="97ac5484-4613-4022-984a-d34327ae5661" containerName="istio-proxy"
Nov 04 03:02:30.559906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.559900    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1586e501-e7c0-4815-97f7-11b953664ef1" containerName="maas-api"
Nov 04 03:02:30.563718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.563702    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.617312 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.617279    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-551e2b-serving-cert\""
Nov 04 03:02:30.681640 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.681059    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:30.697106 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.687962    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.697463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.697431    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.705956 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.697800    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.706341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.706307    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnnw2\" (UniqueName: \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.807791 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.807112    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:02:30.830229 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.830037    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:30.840313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.840270    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnnw2\" (UniqueName: \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.840500 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.840471    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.840569 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.840526    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.840783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.840692    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.841612 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.841585    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.842048 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:30.842029    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-serving-cert: secret "maas-api-e2e-shared-a-w1-551e2b-serving-cert" not found
Nov 04 03:02:30.842178 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:30.842166    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls podName:0916fa7a-82e9-45ad-aa56-2e7b9d34e12a nodeName:}" failed. No retries permitted until 2026-11-04 03:02:31.342145435 +0000 UTC m=+1706.483158820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls") pod "maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" (UID: "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a") : secret "maas-api-e2e-shared-a-w1-551e2b-serving-cert" not found
Nov 04 03:02:30.850823 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.849618    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.905057 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.905028    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnnw2\" (UniqueName: \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:30.975538 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:30.975476    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:31.151150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.140597    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:31.414634 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.414549    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:31.417255 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.417231    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:31.479131 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.479104    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:31.864538 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.864509    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:02:31.866261 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:02:31.866214    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0916fa7a_82e9_45ad_aa56_2e7b9d34e12a.slice/crio-fe2cf7c91e3eb06cc56058d481491968a662c19ba473a9ad0f25351163453f46 WatchSource:0}: Error finding container fe2cf7c91e3eb06cc56058d481491968a662c19ba473a9ad0f25351163453f46: Status 404 returned error can't find the container with id fe2cf7c91e3eb06cc56058d481491968a662c19ba473a9ad0f25351163453f46
Nov 04 03:02:31.868378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:31.868354    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:02:32.693757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:32.693733    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" event={"ID":"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a","Type":"ContainerStarted","Data":"fe2cf7c91e3eb06cc56058d481491968a662c19ba473a9ad0f25351163453f46"}
Nov 04 03:02:33.702617 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:33.702579    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" event={"ID":"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a","Type":"ContainerStarted","Data":"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"}
Nov 04 03:02:33.703087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:33.702664    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:33.820600 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:33.820550    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" podStartSLOduration=3.001007828 podStartE2EDuration="3.820537385s" totalImagesPullingTime="819.529557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:02:33.818512397 +0000 UTC m=+1708.959525817" watchObservedRunningTime="2026-11-04 03:02:33.820537385 +0000 UTC m=+1708.961550792"
Nov 04 03:02:35.297314 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:35.297273    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:36.354718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:36.354672    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:36.604181 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:36.604148    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:02:37.365298 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:37.365266    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:38.033763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:38.033736    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:39.715036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:39.715003    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:02:40.876977 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:40.876941    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:02:40.877437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:40.877208    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" podUID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" containerName="maas-api" containerID="cri-o://518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3" gracePeriod=30
Nov 04 03:02:40.975387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:40.975346    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:41.337399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.337372    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:02:41.404093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.403165    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\") pod \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") "
Nov 04 03:02:41.404093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.403227    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\") pod \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") "
Nov 04 03:02:41.404093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.403318    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\" (UniqueName: \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\") pod \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") "
Nov 04 03:02:41.404093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.403393    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") pod \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\" (UID: \"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381\") "
Nov 04 03:02:41.420152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.405141    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca" pod "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" (UID: "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:02:41.464102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.454218    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls" pod "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" (UID: "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:02:41.464323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.464264    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs" pod "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" (UID: "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:02:41.480100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.465909    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb" pod "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" (UID: "7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381"). InnerVolumeSpecName "kube-api-access-554jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:02:41.512149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.512043    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:41.512149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.512101    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:41.512149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.512117    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-554jb\" (UniqueName: \"kubernetes.io/projected/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-kube-api-access-554jb\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:41.512149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.512131    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:02:41.755585 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.755555    6019 generic.go:356] "Generic (PLEG): container finished" podID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" containerID="518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3" exitCode=0
Nov 04 03:02:41.755754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.755599    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"
Nov 04 03:02:41.755754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.755616    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" event={"ID":"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381","Type":"ContainerDied","Data":"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"}
Nov 04 03:02:41.755754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.755659    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m" event={"ID":"7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381","Type":"ContainerDied","Data":"23702d8aa8ebdaaf8c5efc995aaf87cb175a347d8537a55a973ab9397406d130"}
Nov 04 03:02:41.755754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.755680    6019 scope.go:122] "RemoveContainer" containerID="518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"
Nov 04 03:02:41.767430 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.767412    6019 scope.go:122] "RemoveContainer" containerID="518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"
Nov 04 03:02:41.767697 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:41.767678    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3\": container with ID starting with 518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3 not found: ID does not exist" containerID="518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"
Nov 04 03:02:41.767746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.767705    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3"} err="failed to get container status \"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3\": rpc error: code = NotFound desc = could not find container \"518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3\": container with ID starting with 518d52a8e102f8463bf86538a9152231f0f0b56c207926fed57a8410cdbb71e3 not found: ID does not exist"
Nov 04 03:02:41.920185 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.920154    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:02:42.000097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:41.993086    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db4d0787-799cfb8c66-hjz4m"]
Nov 04 03:02:42.822999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.822971    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:02:42.823615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.823602    6019 cpu_manager.go:414] "containerMap: removing container" podUID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" containerName="maas-api"
Nov 04 03:02:42.823658 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.823618    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" containerName="maas-api"
Nov 04 03:02:42.823744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.823733    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" containerName="maas-api"
Nov 04 03:02:42.828373 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.828357    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:42.852826 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.852792    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:42.852939 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.852909    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:42.853015 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.852948    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:42.853206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.853186    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8jps\" (UniqueName: \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:42.889633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.885312    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:02:42.889633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:42.885333    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-4b2bf4-serving-cert\""
Nov 04 03:02:43.031185 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.020980    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.045379    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.045483    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.045516    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.045720    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8jps\" (UniqueName: \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:43.046269    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-serving-cert: secret "maas-api-e2e-shared-b-w5-4b2bf4-serving-cert" not found
Nov 04 03:02:43.063398 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:02:43.046352    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls podName:1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3 nodeName:}" failed. No retries permitted until 2026-11-04 03:02:43.546329043 +0000 UTC m=+1718.687342431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls") pod "maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" (UID: "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3") : secret "maas-api-e2e-shared-b-w5-4b2bf4-serving-cert" not found
Nov 04 03:02:43.110180 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.071339    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.161123 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.145104    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8jps\" (UniqueName: \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.289061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.289005    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.316091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.314739    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:43.320979 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.320946    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381" path="/var/lib/kubelet/pods/7fa4cd2a-91fc-4c77-ba34-e58ad3ba4381/volumes"
Nov 04 03:02:43.459788 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.459716    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:43.622763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.622729    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.625439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.625418    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:43.743014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:43.742939    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:44.122912 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:44.122887    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:02:44.124316 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:02:44.124289    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3ab4ae_9eb9_44df_ac53_6f8d0b8e0fa3.slice/crio-8e229379da94810f042867f97ab1cee4cac42eb08d6cfdcbdaa2012200a065ea WatchSource:0}: Error finding container 8e229379da94810f042867f97ab1cee4cac42eb08d6cfdcbdaa2012200a065ea: Status 404 returned error can't find the container with id 8e229379da94810f042867f97ab1cee4cac42eb08d6cfdcbdaa2012200a065ea
Nov 04 03:02:44.850092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:44.849991    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" event={"ID":"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3","Type":"ContainerStarted","Data":"8e229379da94810f042867f97ab1cee4cac42eb08d6cfdcbdaa2012200a065ea"}
Nov 04 03:02:45.857871 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:45.857819    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" event={"ID":"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3","Type":"ContainerStarted","Data":"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"}
Nov 04 03:02:45.858480 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:45.857898    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:48.497883 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:48.497840    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:51.741984 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:51.741953    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:51.870728 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:51.870702    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:02:51.887059 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:51.887031    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:51.988849 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:51.988801    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" podStartSLOduration=9.174998401 podStartE2EDuration="9.988788797s" totalImagesPullingTime="813.790396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:02:46.000276199 +0000 UTC m=+1721.141289632" watchObservedRunningTime="2026-11-04 03:02:51.988788797 +0000 UTC m=+1727.129802204"
Nov 04 03:02:53.443148 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:53.443109    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:53.798133 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:53.794740    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:54.236702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:54.236586    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:55.043904 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:55.043872    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:55.238451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:55.238412    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:56.757171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:56.757137    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:57.140159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:57.137789    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:02:59.083420 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:02:59.083385    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:01.810037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:01.810002    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:01.996810 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:01.996781    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:02.218101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:02.217983    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:04.553976 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:04.553877    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:06.604061 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:03:06.604022    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:03:07.351724 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:07.351686    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:07.581359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:07.581321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:09.916986 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:09.916907    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:15.288664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:15.288627    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:20.625601 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:20.625566    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:25.976098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:25.964192    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:27.355830 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:27.355794    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:31.128659 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:31.128624    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:33.434741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:33.434705    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:35.112632 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:35.112588    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:35.695639 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:35.695601    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:36.604817 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:03:36.604776    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:03:36.931950 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:36.931853    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:37.744330 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:37.744298    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:38.916243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:38.916203    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:39.232281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:39.232183    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 03:03:39.232596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:39.232434    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" containerID="cri-o://e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35" gracePeriod=30
Nov 04 03:03:40.304334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:40.304293    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:40.474318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:40.474282    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:40.761379 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:40.761148    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:42.213699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.213666    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:03:42.218359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.218333    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.259377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.259346    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.259563 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.259417    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7kk\" (UniqueName: \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.259644 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.259624    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.259697 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.259663    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.277350 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.277319    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:03:42.323437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.323387    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-569fb7-serving-cert\""
Nov 04 03:03:42.361041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.361000    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7kk\" (UniqueName: \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.361289 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.361263    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.361369 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.361313    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.361438 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.361412    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.361523 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:03:42.361501    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-serving-cert: secret "maas-api-e2e-api-a-569fb7-serving-cert" not found
Nov 04 03:03:42.361601 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:03:42.361588    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls podName:84eff151-b358-4a70-b27c-4c52f6b9d381 nodeName:}" failed. No retries permitted until 2026-11-04 03:03:42.861565571 +0000 UTC m=+1778.002578959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls") pod "maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" (UID: "84eff151-b358-4a70-b27c-4c52f6b9d381") : secret "maas-api-e2e-api-a-569fb7-serving-cert" not found
Nov 04 03:03:42.362343 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.362305    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.364720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.364695    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.416031 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.415997    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:42.422785 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.422760    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7kk\" (UniqueName: \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.641463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.641424    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:42.800104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.797934    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:42.875787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.875743    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:42.878879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:42.878845    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") pod \"maas-api-e2e-api-a-569fb7-88c8b6848-fbthg\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:43.133264 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:43.133230    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:43.216446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:43.216006    6019 patch_prober.go:29] interesting pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:03:43.216446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:43.216084    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:03:43.516655 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:43.516615    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:03:43.522453 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:03:43.522425    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84eff151_b358_4a70_b27c_4c52f6b9d381.slice/crio-897974601b958f50c07ba5a09c8d4ba63d2dafe2a2fefc529a0fb7ce4690a8f6 WatchSource:0}: Error finding container 897974601b958f50c07ba5a09c8d4ba63d2dafe2a2fefc529a0fb7ce4690a8f6: Status 404 returned error can't find the container with id 897974601b958f50c07ba5a09c8d4ba63d2dafe2a2fefc529a0fb7ce4690a8f6
Nov 04 03:03:43.625949 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:43.625908    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" event={"ID":"84eff151-b358-4a70-b27c-4c52f6b9d381","Type":"ContainerStarted","Data":"897974601b958f50c07ba5a09c8d4ba63d2dafe2a2fefc529a0fb7ce4690a8f6"}
Nov 04 03:03:44.563607 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.563228    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.608592    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.608688    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.608750    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.608858    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.608940    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.609038    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.610178    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.610275    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.610366    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\") pod \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\" (UID: \"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3\") "
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.611884    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:03:44.615098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.612550    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:03:44.619284 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.618653    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:44.619284 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.618894    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:03:44.619284 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.619051    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:03:44.642148 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.629035    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:03:44.642148 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.630536    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:03:44.642148 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.640741    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "kube-api-access-nl5jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:03:44.647117 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.645647    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:03:44.656459 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.656419    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" event={"ID":"84eff151-b358-4a70-b27c-4c52f6b9d381","Type":"ContainerStarted","Data":"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"}
Nov 04 03:03:44.657570 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.657539    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo" pod "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" (UID: "1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:03:44.657896 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.657874    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:44.661262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.661230    6019 generic.go:356] "Generic (PLEG): container finished" podID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerID="e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35" exitCode=0
Nov 04 03:03:44.661393 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.661322    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" event={"ID":"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3","Type":"ContainerDied","Data":"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"}
Nov 04 03:03:44.661393 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.661370    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27" event={"ID":"1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3","Type":"ContainerDied","Data":"b02a55bbc6d79094ec66d6516c4dd44410a2969a5d6776246c838f913796ae23"}
Nov 04 03:03:44.661508 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.661400    6019 scope.go:122] "RemoveContainer" containerID="e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"
Nov 04 03:03:44.661601 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.661567    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"
Nov 04 03:03:44.676095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.676061    6019 scope.go:122] "RemoveContainer" containerID="e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"
Nov 04 03:03:44.676391 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:03:44.676368    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35\": container with ID starting with e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35 not found: ID does not exist" containerID="e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"
Nov 04 03:03:44.676472 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.676400    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35"} err="failed to get container status \"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35\": rpc error: code = NotFound desc = could not find container \"e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35\": container with ID starting with e565de7d022108217b60f52f2e0c6ea62a590400a94c656d80ed8fd009245f35 not found: ID does not exist"
Nov 04 03:03:44.713816 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713782    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.713816 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713819    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl5jm\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-kube-api-access-nl5jm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713838    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713853    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713875    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713887    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713898    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713906    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.714092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.713915    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:03:44.792524 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.792428    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:44.830155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.830093    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" podStartSLOduration=1.95691939 podStartE2EDuration="2.830054094s" totalImagesPullingTime="873.134704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:03:44.829280613 +0000 UTC m=+1779.970294059" watchObservedRunningTime="2026-11-04 03:03:44.830054094 +0000 UTC m=+1779.971067502"
Nov 04 03:03:44.945546 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.945511    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 03:03:44.949593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.949564    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27"]
Nov 04 03:03:44.952041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.952013    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:44.984854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:44.984809    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" path="/var/lib/kubelet/pods/1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3/volumes"
Nov 04 03:03:47.873409 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:47.873372    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:48.177887 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:48.177787    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:48.352102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:48.345055    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:51.714992 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:51.714960    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:03:54.213493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:54.213459    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:57.876684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:57.876642    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:58.356841 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:58.356793    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:59.068098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:59.047835    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:59.541422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:59.540768    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:03:59.852096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:03:59.851943    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:02.949002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:02.948968    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:03.125965 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:03.125933    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:03.273064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:03.273030    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:03.529167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:03.529083    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:03.980248 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:03.980145    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:04.568695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.568656    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:04:04.569720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.569691    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy"
Nov 04 03:04:04.569951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.569925    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy"
Nov 04 03:04:04.570247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.570227    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aa9c31e-9c6c-4c24-97a9-a395cb5a83c3" containerName="istio-proxy"
Nov 04 03:04:04.580408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.580379    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.666098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.651640    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-d2b8ee-openshift-default-dockercfg-bd27j\""
Nov 04 03:04:04.692357 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.661540    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:04.826155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.802866    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.826512 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.826483    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.826709 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.826682    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.827039 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.827021    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.827266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.827221    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.872104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.867189    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r224\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.872104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.867293    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.872104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.867370    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.872104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.867449    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:04.953786 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:04.953702    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:04:05.068095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.068035    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.074813 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.074784    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.075505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.075477    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.076343 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.076286    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.076691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.076667    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.077456 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.077438    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.077945 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.077926    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r224\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.078472 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.078452    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.078964 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.078929    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.079551 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.077364    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.081176 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.074605    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.081303 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.078827    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.081388 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.077107    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.081475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.079338    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.101684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.101440    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.168100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.158001    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.168100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.163918    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:05.240112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.228818    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r224\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.240112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.235565    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\") pod \"e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") " pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.301220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.300691    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-d2b8ee-openshift-default-dockercfg-bd27j\""
Nov 04 03:04:05.301220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.300852    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:05.318179 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.317584    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:05.791873 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.791838    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:04:05.793613 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:04:05.793575    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6dbeb7_e0be_45b7_9528_1b84c8dd1fb1.slice/crio-1e327b8864265c2018b101a545ec7cc8bd705c3c044ffa0098cfea3bd7f46e14 WatchSource:0}: Error finding container 1e327b8864265c2018b101a545ec7cc8bd705c3c044ffa0098cfea3bd7f46e14: Status 404 returned error can't find the container with id 1e327b8864265c2018b101a545ec7cc8bd705c3c044ffa0098cfea3bd7f46e14
Nov 04 03:04:05.796417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.796379    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:04:05.796528 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.796459    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:04:05.796528 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.796505    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:04:05.909007 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.908982    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:04:05.921354 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:05.921328    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:04:06.076856 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.076774    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerStarted","Data":"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"}
Nov 04 03:04:06.076856 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.076832    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerStarted","Data":"1e327b8864265c2018b101a545ec7cc8bd705c3c044ffa0098cfea3bd7f46e14"}
Nov 04 03:04:06.204356 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.204294    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podStartSLOduration=2.20427935 podStartE2EDuration="2.20427935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:04:06.201709536 +0000 UTC m=+1801.342722968" watchObservedRunningTime="2026-11-04 03:04:06.20427935 +0000 UTC m=+1801.345292756"
Nov 04 03:04:06.302188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.302155    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:06.304325 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.304299    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:04:06.304447 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:06.304359    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:04:06.605483 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:04:06.605440    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:04:07.306990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:07.306957    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:08.078689 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:08.078651    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:08.089799 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:08.089772    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:08.090692 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:08.090673    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:04:08.326177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:08.326060    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:09.794627 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:09.794564    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:10.065687 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:10.065335    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:10.222029 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:10.221994    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:10.622109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:10.622037    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:15.579748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:15.579719    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:15.887803 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:15.887713    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:17.276309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:17.276275    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:17.745198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:17.745104    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:18.214104 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.186342    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:18.447101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.442440    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:18.586162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.580331    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:04:18.635220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.635143    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:18.651096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.643741    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:04:18.676098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.646227    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.741718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.741665    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.742550 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.742517    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ns7h\" (UniqueName: \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.783382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.783344    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-d2b8ee-dockercfg-685q6\""
Nov 04 03:04:18.783661 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.783640    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-d2b8ee\""
Nov 04 03:04:18.855674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.855574    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ns7h\" (UniqueName: \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.856062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.856042    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.857597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.857575    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:18.918504 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:18.918476    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ns7h\" (UniqueName: \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\") pod \"payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:19.032786 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:19.032752    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:19.438597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:19.434653    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:04:19.516096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:19.515874    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" event={"ID":"0656dde6-e52a-4fce-89d1-aa4fcd4055fe","Type":"ContainerStarted","Data":"78f4628430a94916a52626a220b50d2512b15e3e4622ffa6dd24a75caafe4cf6"}
Nov 04 03:04:20.524016 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:20.523981    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" event={"ID":"0656dde6-e52a-4fce-89d1-aa4fcd4055fe","Type":"ContainerStarted","Data":"5539122b1a8e49f4dab6df3a8f2c5af67f1e6ac752288fa26a509f079b4175ab"}
Nov 04 03:04:20.524486 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:20.524044    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:20.681169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:20.668631    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" podStartSLOduration=2.6686073500000003 podStartE2EDuration="2.66860735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:04:20.639731024 +0000 UTC m=+1815.780744431" watchObservedRunningTime="2026-11-04 03:04:20.66860735 +0000 UTC m=+1815.809620759"
Nov 04 03:04:21.577763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:21.577731    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:22.193742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:22.193703    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:22.373999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:22.373971    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:22.528482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:22.528454    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:27.071216 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:27.071169    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:27.291046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:27.291004    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:27.437819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:27.437133    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:27.726032 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:27.725943    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:28.047098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.040257    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:28.151918 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.151879    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:04:28.166215 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.166184    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.231994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.223567    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:04:28.276101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.264270    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-675c80\""
Nov 04 03:04:28.276101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.264346    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-675c80-dockercfg-hkd97\""
Nov 04 03:04:28.279103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.278487    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.279103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.278982    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbc8\" (UniqueName: \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.397147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.392162    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbc8\" (UniqueName: \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.397147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.392322    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.397147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.393643    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.535095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.526776    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbc8\" (UniqueName: \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\") pod \"payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:28.814937 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:28.814909    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:29.195691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:29.195653    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:04:29.197453 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:04:29.197424    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac5af9f_f63e_4f8a_9a3d_f57bf26a9d1c.slice/crio-9cf193b43ce7c05c54b413f8d3970e9763679129f0f2eb1ada7189dccbbff855 WatchSource:0}: Error finding container 9cf193b43ce7c05c54b413f8d3970e9763679129f0f2eb1ada7189dccbbff855: Status 404 returned error can't find the container with id 9cf193b43ce7c05c54b413f8d3970e9763679129f0f2eb1ada7189dccbbff855
Nov 04 03:04:29.586633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:29.586595    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" event={"ID":"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c","Type":"ContainerStarted","Data":"dc3ccc7ce80b529cb9e1fe3586608cca4bea7fc04495fa227469fed788206bba"}
Nov 04 03:04:29.586855 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:29.586663    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" event={"ID":"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c","Type":"ContainerStarted","Data":"9cf193b43ce7c05c54b413f8d3970e9763679129f0f2eb1ada7189dccbbff855"}
Nov 04 03:04:29.586855 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:29.586694    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:29.704176 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:29.704123    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" podStartSLOduration=1.704109856 podStartE2EDuration="1.704109856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:04:29.701383116 +0000 UTC m=+1824.842396536" watchObservedRunningTime="2026-11-04 03:04:29.704109856 +0000 UTC m=+1824.845123264"
Nov 04 03:04:31.535279 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:31.535244    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:04:31.823959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:31.823838    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:32.406990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:32.406934    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:36.088698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:36.088645    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:36.606001 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:04:36.605961    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:04:37.408051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:37.408019    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:38.107595 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:38.107550    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:40.628843 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:40.628765    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:04:42.418710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:42.418679    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:47.492265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:47.492205    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:52.386720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:52.386607    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:57.230539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:57.230492    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:57.928629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:57.928592    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:04:58.072338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:04:58.072304    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:02.366105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:02.366042    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:06.606442 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:06.606396    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:05:07.278493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:07.278459    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:08.846574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:08.846526    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:09.263589 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:09.263537    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:12.026796 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.026763    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:05:12.031318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.031298    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.083418 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.083396    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cdb11d-serving-cert\""
Nov 04 03:05:12.127137 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.127099    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.127371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.127278    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcszh\" (UniqueName: \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.127536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.127514    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.127625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.127606    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.136764 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.136733    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:05:12.224841 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.224802    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:12.229294 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.229272    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.229406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.229392    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.229453 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:12.229408    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-serving-cert: secret "maas-api-e2e-api-a-cdb11d-serving-cert" not found
Nov 04 03:05:12.229453 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.229448    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcszh\" (UniqueName: \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.229562 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:12.229480    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls podName:538f326a-f939-4ee1-a9ce-a5836d04be99 nodeName:}" failed. No retries permitted until 2026-11-04 03:05:12.729464891 +0000 UTC m=+1867.870478275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls") pod "maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" (UID: "538f326a-f939-4ee1-a9ce-a5836d04be99") : secret "maas-api-e2e-api-a-cdb11d-serving-cert" not found
Nov 04 03:05:12.229628 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.229605    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.230098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.230058    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.232187 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.232167    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.334921 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.334836    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcszh\" (UniqueName: \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.394332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.394294    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:12.554386 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.554348    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:12.740711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.740633    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.743344 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.743321    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") pod \"maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:12.945727 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:12.945696    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:13.329444 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:13.329416    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:05:13.331049 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:05:13.331009    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538f326a_f939_4ee1_a9ce_a5836d04be99.slice/crio-02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc WatchSource:0}: Error finding container 02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc: Status 404 returned error can't find the container with id 02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc
Nov 04 03:05:13.723397 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:13.723293    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:14.000316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:14.000280    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" event={"ID":"538f326a-f939-4ee1-a9ce-a5836d04be99","Type":"ContainerStarted","Data":"02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc"}
Nov 04 03:05:14.539537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:14.539480    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:15.023539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:15.023478    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" event={"ID":"538f326a-f939-4ee1-a9ce-a5836d04be99","Type":"ContainerStarted","Data":"7c504bbd2c5b36de00d487ece680e8627ffeb02bb790baecd05834d2cc7fec0b"}
Nov 04 03:05:15.023704 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:15.023573    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:15.175097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:15.172168    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" podStartSLOduration=3.343261905 podStartE2EDuration="4.172147036s" totalImagesPullingTime="828.885131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:05:15.141503689 +0000 UTC m=+1870.282517096" watchObservedRunningTime="2026-11-04 03:05:15.172147036 +0000 UTC m=+1870.313160447"
Nov 04 03:05:15.181095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:15.178677    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:15.937034 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:15.936998    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:17.912593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:17.912553    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:19.809995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:19.809962    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:19.980225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:19.980197    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:20.129265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:20.129180    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:21.034882 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:21.034854    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:05:23.189454 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:23.189422    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:26.685719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:26.685685    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:26.903780 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:26.903718    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:28.521295 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:28.521261    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:30.343180 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:30.343143    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:33.578231 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.578196    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:05:33.582670 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.582652    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.608804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.608777    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs4tq\" (UniqueName: \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.608918 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.608892    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.608968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.608925    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.609124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.609108    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.677278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.677247    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-00c846-serving-cert\""
Nov 04 03:05:33.680823 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.680796    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:05:33.710322 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.710285    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.710461 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.710328    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.710461 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:33.710414    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-serving-cert: secret "maas-api-e2e-shared-b-w2-00c846-serving-cert" not found
Nov 04 03:05:33.710556 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:33.710474    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls podName:a8bb3939-e67b-4b37-91ce-d384f2021f95 nodeName:}" failed. No retries permitted until 2026-11-04 03:05:34.210460835 +0000 UTC m=+1889.351474221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls") pod "maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" (UID: "a8bb3939-e67b-4b37-91ce-d384f2021f95") : secret "maas-api-e2e-shared-b-w2-00c846-serving-cert" not found
Nov 04 03:05:33.710556 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.710536    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.710674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.710610    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4tq\" (UniqueName: \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.711094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.711054    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.713755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.713733    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.771653 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.771625    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs4tq\" (UniqueName: \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:33.772566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.772534    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:33.940413 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:33.940324    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:34.084169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:34.084132    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:34.237435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:34.237354    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:34.239961 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:34.239933    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:34.496793 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:34.496705    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:34.877592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:34.877566    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:05:34.879059 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:05:34.879036    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8bb3939_e67b_4b37_91ce_d384f2021f95.slice/crio-57c950459094150f6e80a65d4e6d11c59bbc1f211088ea526e16f83759f9b6e6 WatchSource:0}: Error finding container 57c950459094150f6e80a65d4e6d11c59bbc1f211088ea526e16f83759f9b6e6: Status 404 returned error can't find the container with id 57c950459094150f6e80a65d4e6d11c59bbc1f211088ea526e16f83759f9b6e6
Nov 04 03:05:35.271246 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:35.271209    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" event={"ID":"a8bb3939-e67b-4b37-91ce-d384f2021f95","Type":"ContainerStarted","Data":"57c950459094150f6e80a65d4e6d11c59bbc1f211088ea526e16f83759f9b6e6"}
Nov 04 03:05:36.280472 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:36.280428    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" event={"ID":"a8bb3939-e67b-4b37-91ce-d384f2021f95","Type":"ContainerStarted","Data":"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"}
Nov 04 03:05:36.280958 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:36.280495    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:36.406338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:36.406270    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" podStartSLOduration=2.506007194 podStartE2EDuration="3.406247846s" totalImagesPullingTime="900.240652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:05:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:05:36.397530494 +0000 UTC m=+1891.538543926" watchObservedRunningTime="2026-11-04 03:05:36.406247846 +0000 UTC m=+1891.547261256"
Nov 04 03:05:36.408413 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:36.408370    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:36.556157 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:36.555954    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:36.622988 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:36.622947    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:05:38.463438 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:38.463380    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:39.236158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:39.236123    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:42.293486 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:42.293461    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:05:44.139226 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:44.139194    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:44.305317 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:44.305290    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:44.478178 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:44.477940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:49.228717 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:49.228680    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:54.223966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:54.223936    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:58.832138 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.832105    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:05:58.836518 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.836502    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:58.918346 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.918306    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:58.918534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.918384    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rbg6\" (UniqueName: \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:58.918620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.918563    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:58.918717 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.918694    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:58.939987 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.939957    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-675c80-serving-cert\""
Nov 04 03:05:58.945020 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:58.944995    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:05:59.020549 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.020520    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.020713 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:59.020664    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-serving-cert: secret "maas-api-e2e-inf-a-675c80-serving-cert" not found
Nov 04 03:05:59.020767 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.020666    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.020767 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:05:59.020736    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls podName:2fa77641-f337-4123-94ca-342f9a51fc4d nodeName:}" failed. No retries permitted until 2026-11-04 03:05:59.5207156 +0000 UTC m=+1914.661728989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls") pod "maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" (UID: "2fa77641-f337-4123-94ca-342f9a51fc4d") : secret "maas-api-e2e-inf-a-675c80-serving-cert" not found
Nov 04 03:05:59.020852 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.020778    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbg6\" (UniqueName: \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.020964 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.020946    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.021202 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.021186    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.023513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.023495    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.028291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.028268    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:59.121146 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.112282    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rbg6\" (UniqueName: \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.193435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.193397    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:59.344773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.342525    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:05:59.530271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.530242    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.532640 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.532622    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") pod \"maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:05:59.750147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:05:59.750106    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:06:00.132162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:00.132135    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:06:00.133398 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:06:00.133375    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa77641_f337_4123_94ca_342f9a51fc4d.slice/crio-6f209eed322269ea983df79fcd0175b81caed80af80f4058071c13012f8476ce WatchSource:0}: Error finding container 6f209eed322269ea983df79fcd0175b81caed80af80f4058071c13012f8476ce: Status 404 returned error can't find the container with id 6f209eed322269ea983df79fcd0175b81caed80af80f4058071c13012f8476ce
Nov 04 03:06:00.440182 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:00.440088    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:00.663223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:00.663191    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" event={"ID":"2fa77641-f337-4123-94ca-342f9a51fc4d","Type":"ContainerStarted","Data":"6f209eed322269ea983df79fcd0175b81caed80af80f4058071c13012f8476ce"}
Nov 04 03:06:01.672225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:01.672184    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" event={"ID":"2fa77641-f337-4123-94ca-342f9a51fc4d","Type":"ContainerStarted","Data":"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"}
Nov 04 03:06:01.672677 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:01.672396    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:06:01.840741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:01.840689    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" podStartSLOduration=2.894100525 podStartE2EDuration="3.840674461s" totalImagesPullingTime="946.573936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:06:01.838812629 +0000 UTC m=+1916.979826038" watchObservedRunningTime="2026-11-04 03:06:01.840674461 +0000 UTC m=+1916.981687866"
Nov 04 03:06:04.200365 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:04.200329    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:05.903917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:05.903881    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:06.104554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:06.104521    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:06.605287 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:06:06.605251    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:06:06.786531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:06.786496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:07.684617 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:07.684587    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:06:08.455912 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:08.455882    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:08.987624 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:08.987590    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:09.137744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:09.137703    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:10.369938 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:10.369835    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:12.537567 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:12.537534    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:12.686272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:12.686231    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:14.270397 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:14.270362    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:14.784571 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:14.784538    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:15.576320 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:15.576283    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:17.205300 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:17.205268    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:17.909731 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:17.909693    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:18.435965 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:18.435925    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:20.974125 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:20.974026    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:22.761220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:22.761186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:22.948716 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:22.948651    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:23.105484 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:23.105404    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:24.051477 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:24.051442    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:24.132212 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:24.132182    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:06:24.132400 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:24.132249    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:06:24.211238 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:24.211203    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:26.115298 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.115262    6019 generic.go:356] "Generic (PLEG): container finished" podID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerID="fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a" exitCode=0
Nov 04 03:06:26.115754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.115337    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerDied","Data":"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"}
Nov 04 03:06:26.115817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.115803    6019 scope.go:122] "RemoveContainer" containerID="fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"
Nov 04 03:06:26.117707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.117679    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:06:26.117789 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.117748    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:06:26.117831 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.117808    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:06:26.601211 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:26.601179    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:27.278162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:27.276989    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerStarted","Data":"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"}
Nov 04 03:06:27.315619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:27.315584    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:06:27.315972 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:27.315948    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:27.316152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:27.316125    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:27.867719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:27.867648    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:28.301442 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:28.301413    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:28.301847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:28.301458    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:29.327135 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:29.321484    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:29.327135 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:29.321559    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:30.301879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:30.301843    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:30.302118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:30.301896    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:31.302589 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:31.302550    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:31.303011 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:31.302606    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:31.831433 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:31.831381    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:32.302396 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:32.302360    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 04 03:06:32.302570 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:32.302432    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 04 03:06:33.306261 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:33.306229    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:06:33.306715 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:33.306698    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:06:33.307579 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:33.307556    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:06:36.610601 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:06:36.610553    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:06:36.761737 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:36.761704    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:36.907575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:36.907493    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:38.813138 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:38.813106    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:39.198353 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.198273    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 03:06:39.198529 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.198486    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" containerID="cri-o://aafd2acb02aeeaf82de8c979543fe1dc8760eeaf54131199259dc8d88c51b2a8" gracePeriod=30
Nov 04 03:06:39.248959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.248911    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 03:06:39.249254 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.249193    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" containerID="cri-o://3599b89c92730ee25554828909ba25f02cea9e603570edc1a55b69bd15c9577a" gracePeriod=30
Nov 04 03:06:39.254537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.254506    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 03:06:39.254811 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:39.254758    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" containerID="cri-o://72df9bc35e5406252b36863d219029639a591cf107be404608fb22c821bbbccf" gracePeriod=30
Nov 04 03:06:41.138328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.138293    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:41.138732 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.138349    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:41.248491 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.248456    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 03:06:41.248765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.248654    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" containerID="cri-o://585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c" gracePeriod=30
Nov 04 03:06:41.258775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.258743    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 03:06:41.259011 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.258983    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" containerID="cri-o://8699ee630c550e6db01d98872656098656e6ef2fda5d16f72db5d1e637c35640" gracePeriod=30
Nov 04 03:06:41.718393 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.718329    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:41.833528 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.833483    6019 patch_prober.go:29] interesting pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:41.833833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:41.833792    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:43.041771 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:43.041733    6019 patch_prober.go:29] interesting pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:43.042207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:43.041809    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:43.868786 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:43.868749    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:44.180893 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.180816    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:44.180893 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.180883    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:44.545783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.545751    6019 generic.go:356] "Generic (PLEG): container finished" podID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerID="aafd2acb02aeeaf82de8c979543fe1dc8760eeaf54131199259dc8d88c51b2a8" exitCode=0
Nov 04 03:06:44.546036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.545919    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" event={"ID":"50b50a92-262f-40ac-9eca-0f3e7805bff6","Type":"ContainerDied","Data":"aafd2acb02aeeaf82de8c979543fe1dc8760eeaf54131199259dc8d88c51b2a8"}
Nov 04 03:06:44.548238 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.548213    6019 generic.go:356] "Generic (PLEG): container finished" podID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerID="3599b89c92730ee25554828909ba25f02cea9e603570edc1a55b69bd15c9577a" exitCode=0
Nov 04 03:06:44.548370 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.548349    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" event={"ID":"3bd5be74-a0c4-4baa-a3c9-ea620584601c","Type":"ContainerDied","Data":"3599b89c92730ee25554828909ba25f02cea9e603570edc1a55b69bd15c9577a"}
Nov 04 03:06:44.550414 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.550390    6019 generic.go:356] "Generic (PLEG): container finished" podID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerID="72df9bc35e5406252b36863d219029639a591cf107be404608fb22c821bbbccf" exitCode=0
Nov 04 03:06:44.550521 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.550468    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" event={"ID":"c3bf2f90-fed5-4c52-9089-f17620e4fa07","Type":"ContainerDied","Data":"72df9bc35e5406252b36863d219029639a591cf107be404608fb22c821bbbccf"}
Nov 04 03:06:44.610121 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.609226    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 03:06:44.643233 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.643204    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 03:06:44.692965 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.692900    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741483    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741680    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741724    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741756    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741808    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741848    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.741948    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.742043    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.742124    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\") pod \"50b50a92-262f-40ac-9eca-0f3e7805bff6\" (UID: \"50b50a92-262f-40ac-9eca-0f3e7805bff6\") "
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.742338    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.742873    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.743170    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.743410    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.743496    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.747120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.743540    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:44.748281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.748241    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "kube-api-access-72x9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.749457 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.749433    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.755451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.755426    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:44.756857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.756831    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token" pod "50b50a92-262f-40ac-9eca-0f3e7805bff6" (UID: "50b50a92-262f-40ac-9eca-0f3e7805bff6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.844054 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844013    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844054 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844053    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844121    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844215    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844259    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844294    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844347    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844387    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844441    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844495    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844543    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844566    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844540    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844597    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844612    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844617    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844644    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844703    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.844718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844724    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\") pod \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\" (UID: \"3bd5be74-a0c4-4baa-a3c9-ea620584601c\") "
Nov 04 03:06:44.845334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844754    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.845334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844775    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\") pod \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\" (UID: \"c3bf2f90-fed5-4c52-9089-f17620e4fa07\") "
Nov 04 03:06:44.845334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.844832    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.845334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.845011    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.845528 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.845416    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.845587 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.845547    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.845688 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.845632    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.845889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.845850    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.846269 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846151    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:44.846374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846232    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846335    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846351    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846366    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846381    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50b50a92-262f-40ac-9eca-0f3e7805bff6-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846398    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846412    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846427    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846444    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846458    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846475    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846490    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846503    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846518    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846532    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846547    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72x9t\" (UniqueName: \"kubernetes.io/projected/50b50a92-262f-40ac-9eca-0f3e7805bff6-kube-api-access-72x9t\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.846588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.846562    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50b50a92-262f-40ac-9eca-0f3e7805bff6-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.847687 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.847659    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.848446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848368    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "kube-api-access-t98qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.848554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848417    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:44.848554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848476    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "kube-api-access-hx4l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.848554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848480    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.848770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848754    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:44.848936 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.848914    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token" pod "c3bf2f90-fed5-4c52-9089-f17620e4fa07" (UID: "c3bf2f90-fed5-4c52-9089-f17620e4fa07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:44.849134 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.849112    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo" pod "3bd5be74-a0c4-4baa-a3c9-ea620584601c" (UID: "3bd5be74-a0c4-4baa-a3c9-ea620584601c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948167    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948197    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948210    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t98qp\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-kube-api-access-t98qp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948223    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948235    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948247    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx4l5\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-kube-api-access-hx4l5\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948258    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd5be74-a0c4-4baa-a3c9-ea620584601c-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948822 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948271    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:44.948822 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:44.948283    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bf2f90-fed5-4c52-9089-f17620e4fa07-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:45.558597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.558561    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"
Nov 04 03:06:45.559095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.558568    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k" event={"ID":"50b50a92-262f-40ac-9eca-0f3e7805bff6","Type":"ContainerDied","Data":"64597156dee583cb5d5928c2d88543394b2146bb2d3c80295332ac82130dbe3e"}
Nov 04 03:06:45.559095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.558733    6019 scope.go:122] "RemoveContainer" containerID="aafd2acb02aeeaf82de8c979543fe1dc8760eeaf54131199259dc8d88c51b2a8"
Nov 04 03:06:45.560291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.560271    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"
Nov 04 03:06:45.560439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.560282    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj" event={"ID":"3bd5be74-a0c4-4baa-a3c9-ea620584601c","Type":"ContainerDied","Data":"51acfa8ded4f8714464b185a65809adc62aca93cd80fad3010e878e4824fc5e9"}
Nov 04 03:06:45.562102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.562080    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"
Nov 04 03:06:45.562263 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.562229    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh" event={"ID":"c3bf2f90-fed5-4c52-9089-f17620e4fa07","Type":"ContainerDied","Data":"770ffd3e2d9b34089b1d9498af286b3fad2cc4a1cf2a257521d0f0ca20533460"}
Nov 04 03:06:45.570198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.570182    6019 scope.go:122] "RemoveContainer" containerID="3599b89c92730ee25554828909ba25f02cea9e603570edc1a55b69bd15c9577a"
Nov 04 03:06:45.579845 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.579824    6019 scope.go:122] "RemoveContainer" containerID="72df9bc35e5406252b36863d219029639a591cf107be404608fb22c821bbbccf"
Nov 04 03:06:45.613806 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.613776    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:45.613929 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.613830    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:45.723802 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.723766    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 03:06:45.774062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.774026    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh"]
Nov 04 03:06:45.883763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.883640    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 03:06:45.933333 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.933302    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj"]
Nov 04 03:06:45.999047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:45.999015    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 03:06:46.048621 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.048588    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k"]
Nov 04 03:06:46.574600 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.574568    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 03:06:46.575778 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.575751    6019 generic.go:356] "Generic (PLEG): container finished" podID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerID="8699ee630c550e6db01d98872656098656e6ef2fda5d16f72db5d1e637c35640" exitCode=0
Nov 04 03:06:46.575919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.575824    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" event={"ID":"4b13c4f3-cfc8-49b2-85ec-9b35670397a1","Type":"ContainerDied","Data":"8699ee630c550e6db01d98872656098656e6ef2fda5d16f72db5d1e637c35640"}
Nov 04 03:06:46.578510 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.578484    6019 generic.go:356] "Generic (PLEG): container finished" podID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerID="585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c" exitCode=0
Nov 04 03:06:46.578625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.578531    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"
Nov 04 03:06:46.578625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.578569    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" event={"ID":"6709b764-e7ba-48a6-b730-f46cff868bbd","Type":"ContainerDied","Data":"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"}
Nov 04 03:06:46.578710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.578629    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw" event={"ID":"6709b764-e7ba-48a6-b730-f46cff868bbd","Type":"ContainerDied","Data":"93ff34759ab6df9152ae3a9495a132ed6b7b174ed9477e55782e091345aff791"}
Nov 04 03:06:46.578710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.578660    6019 scope.go:122] "RemoveContainer" containerID="585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"
Nov 04 03:06:46.590403 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.590378    6019 scope.go:122] "RemoveContainer" containerID="585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"
Nov 04 03:06:46.590713 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:06:46.590686    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c\": container with ID starting with 585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c not found: ID does not exist" containerID="585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"
Nov 04 03:06:46.590799 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.590726    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c"} err="failed to get container status \"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c\": rpc error: code = NotFound desc = could not find container \"585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c\": container with ID starting with 585f18abb62cc56860129171e08cfe0f78d5a6cfe8b9001ceacaa4a4f90c1e1c not found: ID does not exist"
Nov 04 03:06:46.640510 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.640477    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 03:06:46.677539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677508    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.677539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677543    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.677793 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677619    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.677793 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677642    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.677793 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677660    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.677793 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677773    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677808    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677870    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.677958    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.678062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678059    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678121    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678131    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678176    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678177    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678233    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.678313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678286    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.678623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678321    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678368    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\") pod \"6709b764-e7ba-48a6-b730-f46cff868bbd\" (UID: \"6709b764-e7ba-48a6-b730-f46cff868bbd\") "
Nov 04 03:06:46.678623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678387    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.678623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678409    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678443    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\") pod \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\" (UID: \"4b13c4f3-cfc8-49b2-85ec-9b35670397a1\") "
Nov 04 03:06:46.678881 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678630    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:46.678881 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678771    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.678987 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.678880    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.679154 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.679057    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.679408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.679311    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.679408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.679327    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.679578 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.679512    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.680161 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680137    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680303 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680286    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6709b764-e7ba-48a6-b730-f46cff868bbd-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680410 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680396    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680510 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680494    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680604 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680591    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680689 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680677    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680774 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680763    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680853 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680843    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.680938 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.680927    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.681017 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.681006    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.682549 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.682516    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "kube-api-access-krj2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:46.682728 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.682547    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:46.682728 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.682596    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:46.682865 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.682720    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:46.682926 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.682573    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:46.683463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.683435    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy" pod "4b13c4f3-cfc8-49b2-85ec-9b35670397a1" (UID: "4b13c4f3-cfc8-49b2-85ec-9b35670397a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.683809 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.683786    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:46.683894 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.683851    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:46.684498 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.684482    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz" pod "6709b764-e7ba-48a6-b730-f46cff868bbd" (UID: "6709b764-e7ba-48a6-b730-f46cff868bbd"). InnerVolumeSpecName "kube-api-access-r8qjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819334    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819372    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819387    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8qjz\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-kube-api-access-r8qjz\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819404    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819417    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819432    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krj2x\" (UniqueName: \"kubernetes.io/projected/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-kube-api-access-krj2x\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819446    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6709b764-e7ba-48a6-b730-f46cff868bbd-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.819939 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.819461    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b13c4f3-cfc8-49b2-85ec-9b35670397a1-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:46.983944 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.983911    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" path="/var/lib/kubelet/pods/3bd5be74-a0c4-4baa-a3c9-ea620584601c/volumes"
Nov 04 03:06:46.984454 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.984439    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" path="/var/lib/kubelet/pods/50b50a92-262f-40ac-9eca-0f3e7805bff6/volumes"
Nov 04 03:06:46.984880 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:46.984866    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" path="/var/lib/kubelet/pods/c3bf2f90-fed5-4c52-9089-f17620e4fa07/volumes"
Nov 04 03:06:47.058943 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.058904    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 03:06:47.103522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.103434    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw"]
Nov 04 03:06:47.587572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.587530    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp" event={"ID":"4b13c4f3-cfc8-49b2-85ec-9b35670397a1","Type":"ContainerDied","Data":"f0a771af75dd1c324f7225c7625245918f20f350630588d753eadeb96c27503c"}
Nov 04 03:06:47.587572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.587559    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"
Nov 04 03:06:47.588159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.587587    6019 scope.go:122] "RemoveContainer" containerID="8699ee630c550e6db01d98872656098656e6ef2fda5d16f72db5d1e637c35640"
Nov 04 03:06:47.803205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.803170    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 03:06:47.852283 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:47.852212    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp"]
Nov 04 03:06:48.983470 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:48.983433    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" path="/var/lib/kubelet/pods/4b13c4f3-cfc8-49b2-85ec-9b35670397a1/volumes"
Nov 04 03:06:48.983965 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:48.983950    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" path="/var/lib/kubelet/pods/6709b764-e7ba-48a6-b730-f46cff868bbd/volumes"
Nov 04 03:06:49.579132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:49.579059    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:49.731717 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:49.731474    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:50.076247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:50.076210    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:50.969091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:50.966631    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:06:50.969091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:50.966888    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerName="maas-api" containerID="cri-o://7c504bbd2c5b36de00d487ece680e8627ffeb02bb790baecd05834d2cc7fec0b" gracePeriod=30
Nov 04 03:06:51.030304 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.030272    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.99:8443/health\": dial tcp 10.132.0.99:8443: connect: connection refused"
Nov 04 03:06:51.050857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.050820    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:51.628611 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.628575    6019 generic.go:356] "Generic (PLEG): container finished" podID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerID="7c504bbd2c5b36de00d487ece680e8627ffeb02bb790baecd05834d2cc7fec0b" exitCode=0
Nov 04 03:06:51.628986 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.628658    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" event={"ID":"538f326a-f939-4ee1-a9ce-a5836d04be99","Type":"ContainerDied","Data":"7c504bbd2c5b36de00d487ece680e8627ffeb02bb790baecd05834d2cc7fec0b"}
Nov 04 03:06:51.628986 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.628725    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk" event={"ID":"538f326a-f939-4ee1-a9ce-a5836d04be99","Type":"ContainerDied","Data":"02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc"}
Nov 04 03:06:51.628986 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.628749    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02f930b2fcb1e293a31926078e043a7f2514fcfed3df514fdf682e3942da2bbc"
Nov 04 03:06:51.639569 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.639542    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:06:51.676333 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.676304    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\" (UniqueName: \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\") pod \"538f326a-f939-4ee1-a9ce-a5836d04be99\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") "
Nov 04 03:06:51.676533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.676401    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\") pod \"538f326a-f939-4ee1-a9ce-a5836d04be99\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") "
Nov 04 03:06:51.676533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.676451    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\") pod \"538f326a-f939-4ee1-a9ce-a5836d04be99\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") "
Nov 04 03:06:51.676533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.676522    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") pod \"538f326a-f939-4ee1-a9ce-a5836d04be99\" (UID: \"538f326a-f939-4ee1-a9ce-a5836d04be99\") "
Nov 04 03:06:51.676886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.676856    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca" pod "538f326a-f939-4ee1-a9ce-a5836d04be99" (UID: "538f326a-f939-4ee1-a9ce-a5836d04be99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:51.677371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.677266    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538f326a-f939-4ee1-a9ce-a5836d04be99-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:51.678776 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.678752    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls" pod "538f326a-f939-4ee1-a9ce-a5836d04be99" (UID: "538f326a-f939-4ee1-a9ce-a5836d04be99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:06:51.678868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.678790    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs" pod "538f326a-f939-4ee1-a9ce-a5836d04be99" (UID: "538f326a-f939-4ee1-a9ce-a5836d04be99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:06:51.678910 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.678885    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh" pod "538f326a-f939-4ee1-a9ce-a5836d04be99" (UID: "538f326a-f939-4ee1-a9ce-a5836d04be99"). InnerVolumeSpecName "kube-api-access-qcszh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:51.780496 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.780454    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:51.780747 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.780727    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcszh\" (UniqueName: \"kubernetes.io/projected/538f326a-f939-4ee1-a9ce-a5836d04be99-kube-api-access-qcszh\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:51.780853 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:51.780839    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/538f326a-f939-4ee1-a9ce-a5836d04be99-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:52.635652 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.635619    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"
Nov 04 03:06:52.803658 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.803620    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:06:52.807494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.807463    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdb11d-bdf88f5d8-g6lrk"]
Nov 04 03:06:52.821637 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.821600    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:52.983351 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.983263    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:52.985373 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:52.985345    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" path="/var/lib/kubelet/pods/538f326a-f939-4ee1-a9ce-a5836d04be99/volumes"
Nov 04 03:06:53.251152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:53.251103    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:54.193873 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:54.193824    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 03:06:54.194382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:54.194084    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" containerID="cri-o://552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66" gracePeriod=30
Nov 04 03:06:56.189738 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:56.189697    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:57.892106 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:57.892034    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:06:58.750699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:58.750665    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:06:58.750849 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:58.750737    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:06:59.486196 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.486128    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 03:06:59.637421 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637388    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637421 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637431    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637657 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637462    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637657 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637600    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637657 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637633    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637677    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637747    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.637808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637674    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:59.637808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637773    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:59.638026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637823    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:59.638026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637863    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.638026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.637922    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\") pod \"12d72228-116e-4804-a8c6-09d84e814427\" (UID: \"12d72228-116e-4804-a8c6-09d84e814427\") "
Nov 04 03:06:59.638244 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638182    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:59.638363 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638338    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:06:59.638676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638658    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.638775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638681    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12d72228-116e-4804-a8c6-09d84e814427-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.638775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638696    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.638775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638710    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.638775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.638723    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.640162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.640135    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:06:59.640611 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.640590    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:06:59.640611 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.640600    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:59.641002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.640988    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx" pod "12d72228-116e-4804-a8c6-09d84e814427" (UID: "12d72228-116e-4804-a8c6-09d84e814427"). InnerVolumeSpecName "kube-api-access-vf4qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:06:59.688152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.687924    6019 generic.go:356] "Generic (PLEG): container finished" podID="12d72228-116e-4804-a8c6-09d84e814427" containerID="552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66" exitCode=0
Nov 04 03:06:59.688152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.688010    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" event={"ID":"12d72228-116e-4804-a8c6-09d84e814427","Type":"ContainerDied","Data":"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"}
Nov 04 03:06:59.688152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.688123    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"
Nov 04 03:06:59.688440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.688240    6019 scope.go:122] "RemoveContainer" containerID="552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"
Nov 04 03:06:59.688440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.688201    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs" event={"ID":"12d72228-116e-4804-a8c6-09d84e814427","Type":"ContainerDied","Data":"7ce54e9de8fad9a374fcaf1671ff8c6f315e83fd99120f077e40174ad07ef13a"}
Nov 04 03:06:59.702235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.702216    6019 scope.go:122] "RemoveContainer" containerID="552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"
Nov 04 03:06:59.702801 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:06:59.702774    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66\": container with ID starting with 552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66 not found: ID does not exist" containerID="552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"
Nov 04 03:06:59.703366 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.703333    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66"} err="failed to get container status \"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66\": rpc error: code = NotFound desc = could not find container \"552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66\": container with ID starting with 552b5c3aea91b64bb9e8c03ac8f2e425084b70beb0cf51f8f9aa4e8dbace8e66 not found: ID does not exist"
Nov 04 03:06:59.740967 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.740884    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12d72228-116e-4804-a8c6-09d84e814427-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.740967 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.740922    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf4qx\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-kube-api-access-vf4qx\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.740967 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.740939    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12d72228-116e-4804-a8c6-09d84e814427-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.740967 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.740957    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12d72228-116e-4804-a8c6-09d84e814427-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:06:59.853884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.853844    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 03:06:59.903190 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:06:59.903155    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs"]
Nov 04 03:07:00.983234 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:00.983193    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12d72228-116e-4804-a8c6-09d84e814427" path="/var/lib/kubelet/pods/12d72228-116e-4804-a8c6-09d84e814427/volumes"
Nov 04 03:07:01.183120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:01.183060    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:01.931097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:01.930192    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:02.146186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:02.125839    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:02.956169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:02.952360    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:06.179360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:06.179326    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:06.350382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:06.350329    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:06.510266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:06.510232    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:06.606124 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:07:06.606057    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:07:07.024259 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:07.024225    6019 scope.go:122] "RemoveContainer" containerID="d0dda2c444c1119f3616d9a664817075280a856b67ebc6b1c9497e1055f6759c"
Nov 04 03:07:07.938401 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:07.938359    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:08.086181 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:08.086145    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:08.387902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:08.387867    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:09.804241 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:09.804202    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:09.957124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:09.954464    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:13.449915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:13.449867    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:13.627246 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:13.627208    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:14.530139 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:14.530053    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:15.257707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:15.257671    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:18.859206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:18.854954    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:21.006671 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:21.006609    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:21.160833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:21.160736    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:21.428815 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:21.428733    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:24.025656 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:24.025627    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:27.548016 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:27.547982    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:27.913061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:27.912978    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:07:28.162140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:28.162111    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:28.305295 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:28.305266    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:29.417540 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:29.417496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:31.352704 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:31.352665    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:32.913845 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:32.913790    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" podUID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" containerName="payload-pre-processing" containerID="cri-o://dc3ccc7ce80b529cb9e1fe3586608cca4bea7fc04495fa227469fed788206bba" gracePeriod=25
Nov 04 03:07:33.036759 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.036728    6019 generic.go:356] "Generic (PLEG): container finished" podID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" containerID="dc3ccc7ce80b529cb9e1fe3586608cca4bea7fc04495fa227469fed788206bba" exitCode=0
Nov 04 03:07:33.036886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.036798    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" event={"ID":"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c","Type":"ContainerDied","Data":"dc3ccc7ce80b529cb9e1fe3586608cca4bea7fc04495fa227469fed788206bba"}
Nov 04 03:07:33.197244 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.197223    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:07:33.347132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.347091    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:33.360436 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.360407    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\") pod \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") "
Nov 04 03:07:33.360563 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.360456    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\" (UniqueName: \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\") pod \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\" (UID: \"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c\") "
Nov 04 03:07:33.360771 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.360748    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume" pod "5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" (UID: "5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:07:33.362865 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.362839    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8" pod "5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" (UID: "5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c"). InnerVolumeSpecName "kube-api-access-lqbc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:07:33.462220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.462147    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:33.462220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:33.462185    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqbc8\" (UniqueName: \"kubernetes.io/projected/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c-kube-api-access-lqbc8\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:34.044945 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.044916    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"
Nov 04 03:07:34.045380 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.044962    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf" event={"ID":"5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c","Type":"ContainerDied","Data":"9cf193b43ce7c05c54b413f8d3970e9763679129f0f2eb1ada7189dccbbff855"}
Nov 04 03:07:34.045380 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.045021    6019 scope.go:122] "RemoveContainer" containerID="dc3ccc7ce80b529cb9e1fe3586608cca4bea7fc04495fa227469fed788206bba"
Nov 04 03:07:34.210589 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.210556    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:07:34.259346 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.259319    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf"]
Nov 04 03:07:34.858448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.858415    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:34.988573 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:34.988536    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" path="/var/lib/kubelet/pods/5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c/volumes"
Nov 04 03:07:36.605155 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:07:36.605120    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:07:37.124355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:37.124325    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:37.305885 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:37.305853    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:39.087915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.087881    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:07:39.088460 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.088145    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" podUID="2fa77641-f337-4123-94ca-342f9a51fc4d" containerName="maas-api" containerID="cri-o://8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb" gracePeriod=30
Nov 04 03:07:39.179597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.179564    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:39.395419 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.393702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:39.536760 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.536428    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:07:39.542954 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.538939    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") pod \"2fa77641-f337-4123-94ca-342f9a51fc4d\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") "
Nov 04 03:07:39.543702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.543663    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\") pod \"2fa77641-f337-4123-94ca-342f9a51fc4d\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") "
Nov 04 03:07:39.559999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.559950    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\" (UniqueName: \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\") pod \"2fa77641-f337-4123-94ca-342f9a51fc4d\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") "
Nov 04 03:07:39.560739 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.560702    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\") pod \"2fa77641-f337-4123-94ca-342f9a51fc4d\" (UID: \"2fa77641-f337-4123-94ca-342f9a51fc4d\") "
Nov 04 03:07:39.562171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.562149    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca" pod "2fa77641-f337-4123-94ca-342f9a51fc4d" (UID: "2fa77641-f337-4123-94ca-342f9a51fc4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:07:39.589440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.589147    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls" pod "2fa77641-f337-4123-94ca-342f9a51fc4d" (UID: "2fa77641-f337-4123-94ca-342f9a51fc4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:07:39.592261 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.592214    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6" pod "2fa77641-f337-4123-94ca-342f9a51fc4d" (UID: "2fa77641-f337-4123-94ca-342f9a51fc4d"). InnerVolumeSpecName "kube-api-access-9rbg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:07:39.626593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.626529    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs" pod "2fa77641-f337-4123-94ca-342f9a51fc4d" (UID: "2fa77641-f337-4123-94ca-342f9a51fc4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:07:39.665145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.664947    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rbg6\" (UniqueName: \"kubernetes.io/projected/2fa77641-f337-4123-94ca-342f9a51fc4d-kube-api-access-9rbg6\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:39.665145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.664991    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa77641-f337-4123-94ca-342f9a51fc4d-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:39.665145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.665006    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:39.665145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:39.665020    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa77641-f337-4123-94ca-342f9a51fc4d-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:07:40.086046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.086002    6019 generic.go:356] "Generic (PLEG): container finished" podID="2fa77641-f337-4123-94ca-342f9a51fc4d" containerID="8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb" exitCode=0
Nov 04 03:07:40.086046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.086044    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"
Nov 04 03:07:40.086329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.086129    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" event={"ID":"2fa77641-f337-4123-94ca-342f9a51fc4d","Type":"ContainerDied","Data":"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"}
Nov 04 03:07:40.086329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.086187    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng" event={"ID":"2fa77641-f337-4123-94ca-342f9a51fc4d","Type":"ContainerDied","Data":"6f209eed322269ea983df79fcd0175b81caed80af80f4058071c13012f8476ce"}
Nov 04 03:07:40.086329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.086211    6019 scope.go:122] "RemoveContainer" containerID="8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"
Nov 04 03:07:40.096582 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.096408    6019 scope.go:122] "RemoveContainer" containerID="8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"
Nov 04 03:07:40.096846 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:07:40.096699    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb\": container with ID starting with 8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb not found: ID does not exist" containerID="8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"
Nov 04 03:07:40.096846 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.096723    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb"} err="failed to get container status \"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb\": rpc error: code = NotFound desc = could not find container \"8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb\": container with ID starting with 8f2d19365782f46d00e55532f7e60e2adf5b45238f0070412562806f09e0a7bb not found: ID does not exist"
Nov 04 03:07:40.250251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.250216    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:07:40.300451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.300419    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-675c80-5dccbdb649-s4gng"]
Nov 04 03:07:40.971980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.971905    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:40.983182 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:40.983147    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fa77641-f337-4123-94ca-342f9a51fc4d" path="/var/lib/kubelet/pods/2fa77641-f337-4123-94ca-342f9a51fc4d/volumes"
Nov 04 03:07:45.891396 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:45.891342    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:48.096413 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:48.096381    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:48.249610 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:48.249568    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:49.711091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:49.711044    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:50.235118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:50.235080    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:51.084864 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:51.084831    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:52.761036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:52.761002    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:56.263833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:56.263795    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:56.430274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:56.430239    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:56.572348 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:56.572270    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:07:59.942728 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:07:59.942659    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:01.031220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:01.031024    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:01.183168 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:01.176507    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:02.002228 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:02.002190    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:05.857220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:05.857181    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:06.604435 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:06.604401    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:08:07.570789 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:07.570756    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:09.466783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:09.466743    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:09.613626 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:09.613593    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:12.605618 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:12.605575    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:17.700808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:17.700767    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:19.227505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:19.227439    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:22.002844 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:22.002804    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:23.781051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:23.781021    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:25.763946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:25.763910    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:27.296928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:27.296893    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:27.441579 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:27.441546    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:29.433114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:29.432766    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:29.579857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:29.579824    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:30.176776 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:30.176723    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:30.321975 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:30.321940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:30.828161 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:30.828119    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:31.578615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:31.578582    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:31.818248 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:31.818218    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:32.386108 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:32.386055    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 03:08:32.386609 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:32.386376    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy" containerID="cri-o://c98653cf7e82af7e30259e2b52363dd59d44dc914d35931917d76e1664682a2d" gracePeriod=30
Nov 04 03:08:32.426420 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:32.426391    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:33.242844 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:33.242812    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:33.388482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:33.388449    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:34.010124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:33.998175    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:34.320347 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:34.320249    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:34.640858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:34.640779    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:34.788271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:34.788239    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:34.954151 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:34.942854    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:08:34.960169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:34.954820    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" podUID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" containerName="maas-api" containerID="cri-o://b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756" gracePeriod=30
Nov 04 03:08:35.066097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.048709    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:35.378452 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:35.378414    6019 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0916fa7a_82e9_45ad_aa56_2e7b9d34e12a.slice/crio-b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0916fa7a_82e9_45ad_aa56_2e7b9d34e12a.slice/crio-conmon-b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:08:35.586345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.586274    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:08:35.608525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.608489    6019 generic.go:356] "Generic (PLEG): container finished" podID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" containerID="b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756" exitCode=0
Nov 04 03:08:35.608702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.608549    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"
Nov 04 03:08:35.608702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.608552    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" event={"ID":"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a","Type":"ContainerDied","Data":"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"}
Nov 04 03:08:35.608702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.608606    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j" event={"ID":"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a","Type":"ContainerDied","Data":"fe2cf7c91e3eb06cc56058d481491968a662c19ba473a9ad0f25351163453f46"}
Nov 04 03:08:35.608702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.608634    6019 scope.go:122] "RemoveContainer" containerID="b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"
Nov 04 03:08:35.620576 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.620562    6019 scope.go:122] "RemoveContainer" containerID="b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"
Nov 04 03:08:35.620852 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:35.620834    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756\": container with ID starting with b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756 not found: ID does not exist" containerID="b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"
Nov 04 03:08:35.620900 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.620863    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756"} err="failed to get container status \"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756\": rpc error: code = NotFound desc = could not find container \"b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756\": container with ID starting with b30d1320639fdec7f567045036cd5092b68289a6d5263be8b80d01cd905fe756 not found: ID does not exist"
Nov 04 03:08:35.624318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.624299    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\") pod \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") "
Nov 04 03:08:35.624387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.624360    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") pod \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") "
Nov 04 03:08:35.624470 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.624459    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\") pod \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") "
Nov 04 03:08:35.624501 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.624482    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\" (UniqueName: \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\") pod \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\" (UID: \"0916fa7a-82e9-45ad-aa56-2e7b9d34e12a\") "
Nov 04 03:08:35.624877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.624845    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca" pod "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" (UID: "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:08:35.625337 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.625317    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:35.626600 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.626578    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls" pod "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" (UID: "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:08:35.626702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.626684    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs" pod "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" (UID: "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:08:35.627251 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.627230    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2" pod "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" (UID: "0916fa7a-82e9-45ad-aa56-2e7b9d34e12a"). InnerVolumeSpecName "kube-api-access-jnnw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:08:35.734773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.732209    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnnw2\" (UniqueName: \"kubernetes.io/projected/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-kube-api-access-jnnw2\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:35.739588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.739550    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:35.739817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:35.739803    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:36.096113 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.087495    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:08:36.126739 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.126710    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-551e2b-6fd8fd58f5-2b99j"]
Nov 04 03:08:36.556434 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.556401    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:08:36.556996 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.556984    6019 cpu_manager.go:414] "containerMap: removing container" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.556998    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557011    6019 cpu_manager.go:414] "containerMap: removing container" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557017    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557025    6019 cpu_manager.go:414] "containerMap: removing container" podUID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" containerName="payload-pre-processing"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557031    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" containerName="payload-pre-processing"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557047    6019 cpu_manager.go:414] "containerMap: removing container" podUID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" containerName="maas-api"
Nov 04 03:08:36.557053 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557052    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557081    6019 cpu_manager.go:414] "containerMap: removing container" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557086    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557096    6019 cpu_manager.go:414] "containerMap: removing container" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557101    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557116    6019 cpu_manager.go:414] "containerMap: removing container" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557120    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557133    6019 cpu_manager.go:414] "containerMap: removing container" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557137    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557143    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557147    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557165    6019 cpu_manager.go:414] "containerMap: removing container" podUID="2fa77641-f337-4123-94ca-342f9a51fc4d" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557169    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fa77641-f337-4123-94ca-342f9a51fc4d" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557254    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fa77641-f337-4123-94ca-342f9a51fc4d" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557263    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b13c4f3-cfc8-49b2-85ec-9b35670397a1" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557269    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bd5be74-a0c4-4baa-a3c9-ea620584601c" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557275    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ac5af9f-f63e-4f8a-9a3d-f57bf26a9d1c" containerName="payload-pre-processing"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557282    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557291    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="12d72228-116e-4804-a8c6-09d84e814427" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557298    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="50b50a92-262f-40ac-9eca-0f3e7805bff6" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557305    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="538f326a-f939-4ee1-a9ce-a5836d04be99" containerName="maas-api"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557311    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3bf2f90-fed5-4c52-9089-f17620e4fa07" containerName="istio-proxy"
Nov 04 03:08:36.557332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.557317    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="6709b764-e7ba-48a6-b730-f46cff868bbd" containerName="istio-proxy"
Nov 04 03:08:36.562031 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.562012    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.605707 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:36.605664    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:08:36.615756 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.615692    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-22e72e-serving-cert\""
Nov 04 03:08:36.619467 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.619437    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:08:36.655696 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.655657    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpbwj\" (UniqueName: \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.655696 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.655698    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.655970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.655738    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.655970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.655929    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.754982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.754941    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:36.757566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.757541    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpbwj\" (UniqueName: \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.757650 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.757579    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.757650 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.757627    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.757779 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.757762    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.757858 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:36.757823    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-serving-cert: secret "maas-api-e2e-inf-a-22e72e-serving-cert" not found
Nov 04 03:08:36.757925 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:36.757894    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls podName:e7fdc64a-59ac-4958-a908-fa164927652f nodeName:}" failed. No retries permitted until 2026-11-04 03:08:37.257878989 +0000 UTC m=+2072.398892377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls") pod "maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" (UID: "e7fdc64a-59ac-4958-a908-fa164927652f") : secret "maas-api-e2e-inf-a-22e72e-serving-cert" not found
Nov 04 03:08:36.758398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.758373    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.760513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.760488    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.861156 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.855837    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpbwj\" (UniqueName: \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:36.969138 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.969046    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:36.983242 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:36.983215    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0916fa7a-82e9-45ad-aa56-2e7b9d34e12a" path="/var/lib/kubelet/pods/0916fa7a-82e9-45ad-aa56-2e7b9d34e12a/volumes"
Nov 04 03:08:37.115141 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.114058    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:37.268868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.268834    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:37.271544 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.271515    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") pod \"maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:37.476389 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.476355    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:37.478024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.477993    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:37.630755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.630726    6019 generic.go:356] "Generic (PLEG): container finished" podID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerID="c98653cf7e82af7e30259e2b52363dd59d44dc914d35931917d76e1664682a2d" exitCode=0
Nov 04 03:08:37.630910 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.630775    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerDied","Data":"c98653cf7e82af7e30259e2b52363dd59d44dc914d35931917d76e1664682a2d"}
Nov 04 03:08:37.630910 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.630846    6019 scope.go:122] "RemoveContainer" containerID="35303b9b44f77887bbf90f43e4d154f3f410e1599ebdb623446bc0a1c055e1ad"
Nov 04 03:08:37.714555 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.714532    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:08:37.774753 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.774722    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.774806    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.774844    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.774873    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.774941    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775010    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775131    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775149    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775165    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775249    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.775827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775356    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:08:37.775827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775551    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:08:37.775827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775475    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:08:37.775827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.775289    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\") pod \"ce29f7ec-27de-48f2-9479-c7e0d4712964\" (UID: \"ce29f7ec-27de-48f2-9479-c7e0d4712964\") "
Nov 04 03:08:37.776309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.776280    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.776309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.776305    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.776460 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.776319    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.776460 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.776334    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce29f7ec-27de-48f2-9479-c7e0d4712964-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.776460 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.776349    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.777801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.777777    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:08:37.777917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.777871    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "kube-api-access-dm69k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:08:37.777970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.777962    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:08:37.778014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.777963    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token" pod "ce29f7ec-27de-48f2-9479-c7e0d4712964" (UID: "ce29f7ec-27de-48f2-9479-c7e0d4712964"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:08:37.859194 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.859167    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:08:37.860667 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:08:37.860640    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fdc64a_59ac_4958_a908_fa164927652f.slice/crio-2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1 WatchSource:0}: Error finding container 2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1: Status 404 returned error can't find the container with id 2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1
Nov 04 03:08:37.862169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.862140    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:08:37.877808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.877783    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm69k\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-kube-api-access-dm69k\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.877808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.877810    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.877953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.877819    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:37.877953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:37.877828    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce29f7ec-27de-48f2-9479-c7e0d4712964-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:38.638950 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.638863    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"
Nov 04 03:08:38.639139 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.638867    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67" event={"ID":"ce29f7ec-27de-48f2-9479-c7e0d4712964","Type":"ContainerDied","Data":"d2f4a0afdbd784ea9aa967c68cf7b4bc252c086cb8c3b87f80d96928ba4f12d3"}
Nov 04 03:08:38.639139 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.639042    6019 scope.go:122] "RemoveContainer" containerID="c98653cf7e82af7e30259e2b52363dd59d44dc914d35931917d76e1664682a2d"
Nov 04 03:08:38.640498 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.640200    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" event={"ID":"e7fdc64a-59ac-4958-a908-fa164927652f","Type":"ContainerStarted","Data":"2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1"}
Nov 04 03:08:38.707508 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.707462    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:38.861012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.860981    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 03:08:38.883472 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.883434    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67"]
Nov 04 03:08:38.979865 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.979772    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:38.986979 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:38.986947    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" path="/var/lib/kubelet/pods/ce29f7ec-27de-48f2-9479-c7e0d4712964/volumes"
Nov 04 03:08:39.171098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:39.169911    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:39.607941 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:39.607903    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:39.650037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:39.650006    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" event={"ID":"e7fdc64a-59ac-4958-a908-fa164927652f","Type":"ContainerStarted","Data":"915a8af52f0c74ab578b299f86c26c12848d5edacfb598d4ce9a78802175c1e4"}
Nov 04 03:08:39.650234 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:39.650140    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:39.772247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:39.772185    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" podStartSLOduration=2.883744229 podStartE2EDuration="3.772167749s" totalImagesPullingTime="888.42352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:08:39.769446041 +0000 UTC m=+2074.910459463" watchObservedRunningTime="2026-11-04 03:08:39.772167749 +0000 UTC m=+2074.913181191"
Nov 04 03:08:40.276568 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:40.276529    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:40.420828 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:40.420799    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:40.566394 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:40.566312    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:40.822722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:40.822627    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:41.650940 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:41.650902    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:42.388367 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:42.388336    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:45.663034 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:45.663009    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:08:47.910236 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:47.910188    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:48.054952 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:48.054916    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:48.196182 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:48.196099    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:48.384330 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:48.379648    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:49.533740 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:49.533707    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:08:49.808191 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:49.808095    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:49.961873 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:49.961819    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:52.728237 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:52.728204    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:08:52.728674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:52.728412    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" podUID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" containerName="maas-api" containerID="cri-o://48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40" gracePeriod=30
Nov 04 03:08:52.825768 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:52.825737    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:53.137098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.116766    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:08:53.161097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.138214    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\" (UniqueName: \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\") pod \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") "
Nov 04 03:08:53.162816 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.162777    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\") pod \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") "
Nov 04 03:08:53.174097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.168723    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\") pod \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") "
Nov 04 03:08:53.174360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.174338    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") pod \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\" (UID: \"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3\") "
Nov 04 03:08:53.174489 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.168605    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca" pod "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" (UID: "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:08:53.183098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.175414    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:53.203054 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.182220    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs" pod "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" (UID: "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:08:53.203280 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.203002    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps" pod "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" (UID: "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3"). InnerVolumeSpecName "kube-api-access-k8jps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:08:53.254675 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.254629    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls" pod "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" (UID: "1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:08:53.315484 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.315449    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8jps\" (UniqueName: \"kubernetes.io/projected/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-kube-api-access-k8jps\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:53.316194 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.316160    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:53.316194 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.316196    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:53.472739 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.472651    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:53.623643 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.623611    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:53.862417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.862369    6019 generic.go:356] "Generic (PLEG): container finished" podID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" containerID="48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40" exitCode=0
Nov 04 03:08:53.863223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.863184    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" event={"ID":"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3","Type":"ContainerDied","Data":"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"}
Nov 04 03:08:53.863549 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.863532    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8" event={"ID":"1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3","Type":"ContainerDied","Data":"8e229379da94810f042867f97ab1cee4cac42eb08d6cfdcbdaa2012200a065ea"}
Nov 04 03:08:53.863691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.863680    6019 scope.go:122] "RemoveContainer" containerID="48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"
Nov 04 03:08:53.863897 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.863461    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"
Nov 04 03:08:53.920286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.916703    6019 scope.go:122] "RemoveContainer" containerID="48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"
Nov 04 03:08:53.922730 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:08:53.922646    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40\": container with ID starting with 48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40 not found: ID does not exist" containerID="48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"
Nov 04 03:08:53.922730 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:53.922690    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40"} err="failed to get container status \"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40\": rpc error: code = NotFound desc = could not find container \"48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40\": container with ID starting with 48d654f8afacb933112dc55ac7b80ce2c97f14ca38e2d795d37811ee7ba27d40 not found: ID does not exist"
Nov 04 03:08:54.062254 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.056959    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:08:54.099791 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.099662    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-4b2bf4-7fcf58b4db-l7nk8"]
Nov 04 03:08:54.242588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.242554    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:54.535612 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.535572    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" podUID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" containerName="payload-pre-processing" containerID="cri-o://5539122b1a8e49f4dab6df3a8f2c5af67f1e6ac752288fa26a509f079b4175ab" gracePeriod=25
Nov 04 03:08:54.877153 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.876016    6019 generic.go:356] "Generic (PLEG): container finished" podID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" containerID="5539122b1a8e49f4dab6df3a8f2c5af67f1e6ac752288fa26a509f079b4175ab" exitCode=0
Nov 04 03:08:54.877153 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.876248    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" event={"ID":"0656dde6-e52a-4fce-89d1-aa4fcd4055fe","Type":"ContainerDied","Data":"5539122b1a8e49f4dab6df3a8f2c5af67f1e6ac752288fa26a509f079b4175ab"}
Nov 04 03:08:54.911782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.911758    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:08:54.928423 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.928400    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\" (UniqueName: \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\") pod \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") "
Nov 04 03:08:54.928553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.928539    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\") pod \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\" (UID: \"0656dde6-e52a-4fce-89d1-aa4fcd4055fe\") "
Nov 04 03:08:54.928886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.928861    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume" pod "0656dde6-e52a-4fce-89d1-aa4fcd4055fe" (UID: "0656dde6-e52a-4fce-89d1-aa4fcd4055fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:08:54.930592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.930570    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h" pod "0656dde6-e52a-4fce-89d1-aa4fcd4055fe" (UID: "0656dde6-e52a-4fce-89d1-aa4fcd4055fe"). InnerVolumeSpecName "kube-api-access-5ns7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:08:54.983968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:54.983935    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" path="/var/lib/kubelet/pods/1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3/volumes"
Nov 04 03:08:55.030119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.029880    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:55.030119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.029906    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ns7h\" (UniqueName: \"kubernetes.io/projected/0656dde6-e52a-4fce-89d1-aa4fcd4055fe-kube-api-access-5ns7h\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:08:55.083513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.083489    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:55.397863 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.397818    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:55.884495 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.884466    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"
Nov 04 03:08:55.884927 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.884499    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29" event={"ID":"0656dde6-e52a-4fce-89d1-aa4fcd4055fe","Type":"ContainerDied","Data":"78f4628430a94916a52626a220b50d2512b15e3e4622ffa6dd24a75caafe4cf6"}
Nov 04 03:08:55.884927 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:55.884557    6019 scope.go:122] "RemoveContainer" containerID="5539122b1a8e49f4dab6df3a8f2c5af67f1e6ac752288fa26a509f079b4175ab"
Nov 04 03:08:56.049408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:56.049376    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:08:56.099099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:56.099058    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29"]
Nov 04 03:08:56.176446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:56.176371    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:08:56.983158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:08:56.983130    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" path="/var/lib/kubelet/pods/0656dde6-e52a-4fce-89d1-aa4fcd4055fe/volumes"
Nov 04 03:09:00.888674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:00.888639    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:01.034849 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:01.034813    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:01.181741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:01.181652    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:01.522160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:01.522097    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:01.668532 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:01.668494    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:02.125951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:02.125896    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:02.335692 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:02.335657    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:02.628872 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:02.628805    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:03.103340 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:03.103307    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:09:03.103533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:03.103513    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" containerID="cri-o://31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f" gracePeriod=30
Nov 04 03:09:03.105784 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:03.105748    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:03.328461 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:03.328420    6019 patch_prober.go:29] interesting pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:09:03.329255 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:03.329222    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:09:04.024172 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:04.024139    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:04.550382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:04.550317    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:05.044434 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:05.044391    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:05.955895 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:05.955859    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:09:05.969008 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:05.968987    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:09:06.489235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:06.489195    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:06.633329 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:06.633291    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:09:08.378550 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.378490    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:09:08.438390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438349    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438430    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438495    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438630 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438577    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438630 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438610    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438706 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438655    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438811 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438785    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438893 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438812    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:08.438951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438904    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.438951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438942    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\") pod \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\" (UID: \"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1\") "
Nov 04 03:09:08.439155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438941    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:08.439155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.438971    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:08.439155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439152    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:08.439422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439263    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:08.439637 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439614    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.439707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439642    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.439707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439653    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.439707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439662    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.439707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.439673    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.440837 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.440808    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:08.441000 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.440982    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:09:08.441351 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.441331    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:08.441439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.441406    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224" pod "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" (UID: "0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1"). InnerVolumeSpecName "kube-api-access-8r224". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:08.541274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.541241    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.541274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.541275    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.541495 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.541286    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.541495 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.541296    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r224\" (UniqueName: \"kubernetes.io/projected/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1-kube-api-access-8r224\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:08.970776 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.970737    6019 generic.go:356] "Generic (PLEG): container finished" podID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerID="31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f" exitCode=0
Nov 04 03:09:08.970959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.970788    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"
Nov 04 03:09:08.970959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.970831    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerDied","Data":"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"}
Nov 04 03:09:08.970959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.970884    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9" event={"ID":"0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1","Type":"ContainerDied","Data":"1e327b8864265c2018b101a545ec7cc8bd705c3c044ffa0098cfea3bd7f46e14"}
Nov 04 03:09:08.970959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.970907    6019 scope.go:122] "RemoveContainer" containerID="31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"
Nov 04 03:09:08.985399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.985362    6019 scope.go:122] "RemoveContainer" containerID="fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"
Nov 04 03:09:08.997685 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.997662    6019 scope.go:122] "RemoveContainer" containerID="31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"
Nov 04 03:09:08.998009 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:08.997989    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f\": container with ID starting with 31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f not found: ID does not exist" containerID="31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"
Nov 04 03:09:08.998061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.998020    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f"} err="failed to get container status \"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f\": rpc error: code = NotFound desc = could not find container \"31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f\": container with ID starting with 31a6d8ae6302782da6d80a2c0e5e7ef2eec9bad9d4b419d39373e12289a4f16f not found: ID does not exist"
Nov 04 03:09:08.998061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.998038    6019 scope.go:122] "RemoveContainer" containerID="fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"
Nov 04 03:09:08.998351 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:08.998331    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a\": container with ID starting with fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a not found: ID does not exist" containerID="fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"
Nov 04 03:09:08.998396 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:08.998358    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a"} err="failed to get container status \"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a\": rpc error: code = NotFound desc = could not find container \"fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a\": container with ID starting with fafb818971330f3fcdddac3851e04a43508aae486efc8d689043c8eec3eb2d5a not found: ID does not exist"
Nov 04 03:09:09.137477 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:09.137443    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:09:09.189729 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:09.189693    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9"]
Nov 04 03:09:10.008507 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:10.008424    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 03:09:10.059345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:10.059313    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 03:09:10.202819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:10.202785    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:10.982562 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:10.982521    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" path="/var/lib/kubelet/pods/0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1/volumes"
Nov 04 03:09:11.203877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:11.203845    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:11.347981 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:11.347940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:11.837382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:11.837355    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:12.457326 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:12.457289    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:16.691371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:16.691333    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:18.565955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:18.565917    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:18.898617 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:18.898532    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 03:09:19.264922 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:19.264892    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:19.407563 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:19.407516    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:21.932439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:21.932402    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:22.105132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:22.105096    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:23.899683 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:23.899643    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing" containerID="cri-o://5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098" gracePeriod=25
Nov 04 03:09:24.444773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:24.444736    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:25.032361 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.032321    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main" containerID="cri-o://3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b" gracePeriod=45
Nov 04 03:09:25.083772 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.083731    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main" containerID="cri-o://66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87" gracePeriod=45
Nov 04 03:09:25.381860 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.381828    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 03:09:25.385333 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.385309    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 03:09:25.484648 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484616    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\" (UniqueName: \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.484835 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484671    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.484835 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484727    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\" (UniqueName: \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.484835 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484755    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.485019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484898    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\" (UniqueName: \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.485019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.484954    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.485019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485012    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.485206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485089    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.485206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485144    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\") pod \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\" (UID: \"b70d340e-cc4c-4e8d-8c9d-f4660894d103\") "
Nov 04 03:09:25.485206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485182    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\" (UniqueName: \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.485355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485211    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.485355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485275    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\") pod \"769a6f5a-1811-4106-a8a7-b1636a39218b\" (UID: \"769a6f5a-1811-4106-a8a7-b1636a39218b\") "
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485268    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485330    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485513    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.485670    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.486227    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.486684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.486420    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.487510 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487472    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487510 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487498    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487513    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487526    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487540    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487551    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.487978 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.487890    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.488307 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.488280    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:25.488408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.488363    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:09:25.489490 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.489463    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:09:25.489588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.489507    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc" pod "769a6f5a-1811-4106-a8a7-b1636a39218b" (UID: "769a6f5a-1811-4106-a8a7-b1636a39218b"). InnerVolumeSpecName "kube-api-access-2s8tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:25.489960 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.489941    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf" pod "b70d340e-cc4c-4e8d-8c9d-f4660894d103" (UID: "b70d340e-cc4c-4e8d-8c9d-f4660894d103"). InnerVolumeSpecName "kube-api-access-plxsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:25.588485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588450    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769a6f5a-1811-4106-a8a7-b1636a39218b-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.588485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588480    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70d340e-cc4c-4e8d-8c9d-f4660894d103-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.588485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588489    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70d340e-cc4c-4e8d-8c9d-f4660894d103-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.588708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588507    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plxsf\" (UniqueName: \"kubernetes.io/projected/b70d340e-cc4c-4e8d-8c9d-f4660894d103-kube-api-access-plxsf\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.588708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588520    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769a6f5a-1811-4106-a8a7-b1636a39218b-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:25.588708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:25.588528    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s8tc\" (UniqueName: \"kubernetes.io/projected/769a6f5a-1811-4106-a8a7-b1636a39218b-kube-api-access-2s8tc\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:26.087103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.087054    6019 generic.go:356] "Generic (PLEG): container finished" podID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerID="66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87" exitCode=0
Nov 04 03:09:26.087554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.087105    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" event={"ID":"b70d340e-cc4c-4e8d-8c9d-f4660894d103","Type":"ContainerDied","Data":"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"}
Nov 04 03:09:26.087554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.087132    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"
Nov 04 03:09:26.087554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.087158    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8" event={"ID":"b70d340e-cc4c-4e8d-8c9d-f4660894d103","Type":"ContainerDied","Data":"a713eab3628ff6750d750ec0a71a111e82b8c4d3056b82749b5de32cc1d5cddf"}
Nov 04 03:09:26.087554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.087180    6019 scope.go:122] "RemoveContainer" containerID="66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"
Nov 04 03:09:26.089395 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.089369    6019 generic.go:356] "Generic (PLEG): container finished" podID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerID="3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b" exitCode=0
Nov 04 03:09:26.089533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.089519    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" event={"ID":"769a6f5a-1811-4106-a8a7-b1636a39218b","Type":"ContainerDied","Data":"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"}
Nov 04 03:09:26.089595 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.089548    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt" event={"ID":"769a6f5a-1811-4106-a8a7-b1636a39218b","Type":"ContainerDied","Data":"133937624a0c434abecbb594da9c3d340da5406c25713a76a425bc513021abcc"}
Nov 04 03:09:26.089644 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.089594    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"
Nov 04 03:09:26.098284 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.098267    6019 scope.go:122] "RemoveContainer" containerID="66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"
Nov 04 03:09:26.098590 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:26.098567    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87\": container with ID starting with 66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87 not found: ID does not exist" containerID="66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"
Nov 04 03:09:26.098669 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.098590    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87"} err="failed to get container status \"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87\": rpc error: code = NotFound desc = could not find container \"66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87\": container with ID starting with 66bfa0864bf8dd8c073067ccf81edda46d07717cc14214bc83072f7d236d4e87 not found: ID does not exist"
Nov 04 03:09:26.098669 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.098606    6019 scope.go:122] "RemoveContainer" containerID="3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"
Nov 04 03:09:26.107919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.107900    6019 scope.go:122] "RemoveContainer" containerID="3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"
Nov 04 03:09:26.108252 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:26.108227    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b\": container with ID starting with 3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b not found: ID does not exist" containerID="3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"
Nov 04 03:09:26.108350 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.108256    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b"} err="failed to get container status \"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b\": rpc error: code = NotFound desc = could not find container \"3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b\": container with ID starting with 3d92caa67db47d070d4b5d76a8119ce15da5ad6030fc78441161d3f4fbdf056b not found: ID does not exist"
Nov 04 03:09:26.251024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.250990    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 03:09:26.300524 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.300495    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/distinct-2-w5-3b7476-kserve-799c549f96-qhvzt"]
Nov 04 03:09:26.363465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.363388    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 03:09:26.367327 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.367298    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-3b7476/premium-w5-3b7476-kserve-56cfb88bfb-965k8"]
Nov 04 03:09:26.937355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.937320    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:09:26.983161 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.983125    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" path="/var/lib/kubelet/pods/769a6f5a-1811-4106-a8a7-b1636a39218b/volumes"
Nov 04 03:09:26.983711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:26.983687    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" path="/var/lib/kubelet/pods/b70d340e-cc4c-4e8d-8c9d-f4660894d103/volumes"
Nov 04 03:09:27.499673 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:27.499641    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:28.178101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:28.178055    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:28.730387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:28.730352    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:28.876332 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:28.876277    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:31.054892 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:31.054849    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:32.877023 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:32.876987    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:33.164034 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:33.163696    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:33.630118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:33.627917    6019 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:09:33.654436 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:33.654248    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:09:33.654436 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:33.654354    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:09:33.698239 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:33.698195    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 03:09:34.859437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:34.859382    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:36.604596 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:36.604557    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:09:36.936999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:36.936912    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:09:37.888737 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:37.888698    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:38.033959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:38.033903    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:39.203265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:39.203231    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 03:09:39.203763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:39.203459    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" containerID="cri-o://2ca5ba1bee21d377ed6f5aa2131608f41cd48b724144dd0301ab2f7f2a54620c" gracePeriod=30
Nov 04 03:09:39.253871 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:39.253829    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 03:09:39.254115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:39.254088    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" containerID="cri-o://20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35" gracePeriod=30
Nov 04 03:09:39.968230 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:39.968194    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:40.413808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:40.413772    6019 patch_prober.go:29] interesting pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:09:40.414238 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:40.413841    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:09:42.130708 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:42.130670    6019 patch_prober.go:29] interesting pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:09:42.131164 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:42.130743    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:09:42.293064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:42.293027    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 03:09:42.293296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:42.293270    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" containerID="cri-o://30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10" gracePeriod=30
Nov 04 03:09:42.898273 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:42.898239    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:44.163680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.163619    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 03:09:44.298400 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.298364    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\" (UniqueName: \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\") pod \"76d45294-a924-42cf-8e63-c31279716771\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") "
Nov 04 03:09:44.298528 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.298509    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\") pod \"76d45294-a924-42cf-8e63-c31279716771\" (UID: \"76d45294-a924-42cf-8e63-c31279716771\") "
Nov 04 03:09:44.298984 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.298879    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume" pod "76d45294-a924-42cf-8e63-c31279716771" (UID: "76d45294-a924-42cf-8e63-c31279716771"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:44.299440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.299418    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d45294-a924-42cf-8e63-c31279716771-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.300704 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.300682    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm" pod "76d45294-a924-42cf-8e63-c31279716771" (UID: "76d45294-a924-42cf-8e63-c31279716771"). InnerVolumeSpecName "kube-api-access-js8hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:44.337248 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.337217    6019 generic.go:356] "Generic (PLEG): container finished" podID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerID="2ca5ba1bee21d377ed6f5aa2131608f41cd48b724144dd0301ab2f7f2a54620c" exitCode=0
Nov 04 03:09:44.337375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.337291    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" event={"ID":"bb95735c-4dea-4c41-adbf-2fddd9288f2b","Type":"ContainerDied","Data":"2ca5ba1bee21d377ed6f5aa2131608f41cd48b724144dd0301ab2f7f2a54620c"}
Nov 04 03:09:44.338917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.338896    6019 generic.go:356] "Generic (PLEG): container finished" podID="76d45294-a924-42cf-8e63-c31279716771" containerID="5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098" exitCode=0
Nov 04 03:09:44.339051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.338951    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"
Nov 04 03:09:44.339051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.338985    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" event={"ID":"76d45294-a924-42cf-8e63-c31279716771","Type":"ContainerDied","Data":"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"}
Nov 04 03:09:44.339051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.339035    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw" event={"ID":"76d45294-a924-42cf-8e63-c31279716771","Type":"ContainerDied","Data":"a686a677d366b715dacb02fcfde3047008fc7ba12c34271bf054d53c9c6cd1ee"}
Nov 04 03:09:44.339306 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.339057    6019 scope.go:122] "RemoveContainer" containerID="5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"
Nov 04 03:09:44.367040 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.367020    6019 scope.go:122] "RemoveContainer" containerID="5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"
Nov 04 03:09:44.367383 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:44.367325    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098\": container with ID starting with 5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098 not found: ID does not exist" containerID="5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"
Nov 04 03:09:44.367383 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.367358    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098"} err="failed to get container status \"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098\": rpc error: code = NotFound desc = could not find container \"5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098\": container with ID starting with 5793070bb6f0dc6fb42a5aa97b95d82b355bcbd4422c8c93c625d77f75bcd098 not found: ID does not exist"
Nov 04 03:09:44.400801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.400769    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js8hm\" (UniqueName: \"kubernetes.io/projected/76d45294-a924-42cf-8e63-c31279716771-kube-api-access-js8hm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.505700 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.505667    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 03:09:44.513574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.513552    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw"]
Nov 04 03:09:44.517674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.517647    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 03:09:44.545682 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.545662    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 03:09:44.603132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603103    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603151    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603181    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603227    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603248    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603552 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603358    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603552 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603434    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603552 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603458    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603701 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603565    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\") pod \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\" (UID: \"bb95735c-4dea-4c41-adbf-2fddd9288f2b\") "
Nov 04 03:09:44.603701 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603539    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:44.603701 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603639    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.603916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603717    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.603916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603786    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.604016 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.603954    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.604502 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.604478    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.604560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.604508    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.604560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.604525    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.604560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.604541    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.604560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.604555    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.605768 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.605743    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:09:44.605928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.605755    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.606477 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.606455    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:44.606552 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.606464    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8" pod "bb95735c-4dea-4c41-adbf-2fddd9288f2b" (UID: "bb95735c-4dea-4c41-adbf-2fddd9288f2b"). InnerVolumeSpecName "kube-api-access-565m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:44.705803 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.705711    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.705983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.705822    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.705983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.705897    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.705983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.705958    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.705984    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706061    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706133    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706172    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706183    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.706309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706203    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\") pod \"b6288f05-af04-46c0-bb21-71a943669694\" (UID: \"b6288f05-af04-46c0-bb21-71a943669694\") "
Nov 04 03:09:44.706309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706266    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.706532 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706385    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.706592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706483    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.706862 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706839    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.706993    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707010    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707025    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707038    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707050    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707063    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707095    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288f05-af04-46c0-bb21-71a943669694-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707108    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-565m8\" (UniqueName: \"kubernetes.io/projected/bb95735c-4dea-4c41-adbf-2fddd9288f2b-kube-api-access-565m8\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.707177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.707123    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95735c-4dea-4c41-adbf-2fddd9288f2b-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.708293 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.708271    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "kube-api-access-dnncw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:44.708653 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.708625    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:44.708838 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.708815    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:44.708886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.708818    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo" pod "b6288f05-af04-46c0-bb21-71a943669694" (UID: "b6288f05-af04-46c0-bb21-71a943669694"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:09:44.808540 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.808504    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288f05-af04-46c0-bb21-71a943669694-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.808540 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.808537    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288f05-af04-46c0-bb21-71a943669694-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.808744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.808551    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.808744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.808564    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnncw\" (UniqueName: \"kubernetes.io/projected/b6288f05-af04-46c0-bb21-71a943669694-kube-api-access-dnncw\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:44.984194 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:44.984125    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76d45294-a924-42cf-8e63-c31279716771" path="/var/lib/kubelet/pods/76d45294-a924-42cf-8e63-c31279716771/volumes"
Nov 04 03:09:45.348847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.348809    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"
Nov 04 03:09:45.348847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.348821    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw" event={"ID":"bb95735c-4dea-4c41-adbf-2fddd9288f2b","Type":"ContainerDied","Data":"322d790a733e61273ce5c269c030dae66dc36c659877efca43e244549aed9997"}
Nov 04 03:09:45.349366 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.348886    6019 scope.go:122] "RemoveContainer" containerID="2ca5ba1bee21d377ed6f5aa2131608f41cd48b724144dd0301ab2f7f2a54620c"
Nov 04 03:09:45.350494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.350468    6019 generic.go:356] "Generic (PLEG): container finished" podID="b6288f05-af04-46c0-bb21-71a943669694" containerID="20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35" exitCode=0
Nov 04 03:09:45.350605 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.350531    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"
Nov 04 03:09:45.350605 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.350540    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" event={"ID":"b6288f05-af04-46c0-bb21-71a943669694","Type":"ContainerDied","Data":"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"}
Nov 04 03:09:45.350605 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.350583    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n" event={"ID":"b6288f05-af04-46c0-bb21-71a943669694","Type":"ContainerDied","Data":"6e2e59528e1e176155146c207782bca7827b571d01d89a89f32f9bfdf6e9b759"}
Nov 04 03:09:45.359378 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.359357    6019 scope.go:122] "RemoveContainer" containerID="20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"
Nov 04 03:09:45.368247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.368228    6019 scope.go:122] "RemoveContainer" containerID="20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"
Nov 04 03:09:45.368505 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:45.368482    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35\": container with ID starting with 20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35 not found: ID does not exist" containerID="20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"
Nov 04 03:09:45.368569 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.368514    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35"} err="failed to get container status \"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35\": rpc error: code = NotFound desc = could not find container \"20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35\": container with ID starting with 20213d221795501858056907cb454c4f653f44e5f69b1bae4dae8d8d8a662d35 not found: ID does not exist"
Nov 04 03:09:45.522613 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.522555    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 03:09:45.572205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.572173    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n"]
Nov 04 03:09:45.681827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.681739    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 03:09:45.686364 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:45.686336    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw"]
Nov 04 03:09:46.989280 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:46.989243    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6288f05-af04-46c0-bb21-71a943669694" path="/var/lib/kubelet/pods/b6288f05-af04-46c0-bb21-71a943669694/volumes"
Nov 04 03:09:46.989893 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:46.989873    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" path="/var/lib/kubelet/pods/bb95735c-4dea-4c41-adbf-2fddd9288f2b/volumes"
Nov 04 03:09:47.139098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.134862    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:09:47.188133 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.184628    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:09:47.188133 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.184865    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:09:47.239215 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.239186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:47.385090 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.385025    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" podUID="84eff151-b358-4a70-b27c-4c52f6b9d381" containerName="maas-api" containerID="cri-o://d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b" gracePeriod=30
Nov 04 03:09:47.684141 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.645225    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751430    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751483    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751631    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751695    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751800    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751831    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751866    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751916    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.755096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.751953    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\") pod \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\" (UID: \"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b\") "
Nov 04 03:09:47.768041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.767303    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:47.768041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.767382    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:47.768041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.767967    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:47.769049 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.768678    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:47.769315 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.769282    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:47.792024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.791296    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "kube-api-access-nbw95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:47.814296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.813168    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:47.814296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.813168    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:09:47.814296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.814235    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy" pod "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" (UID: "6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:09:47.825366 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.825280    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:09:47.869087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.868990    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869030    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869047    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869062    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869097    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbw95\" (UniqueName: \"kubernetes.io/projected/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-kube-api-access-nbw95\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869113    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869128    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869143    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.869329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.869155    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.969590 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.969552    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") pod \"84eff151-b358-4a70-b27c-4c52f6b9d381\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") "
Nov 04 03:09:47.969826 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.969617    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\" (UniqueName: \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\") pod \"84eff151-b358-4a70-b27c-4c52f6b9d381\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") "
Nov 04 03:09:47.969826 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.969754    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\") pod \"84eff151-b358-4a70-b27c-4c52f6b9d381\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") "
Nov 04 03:09:47.969945 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.969905    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\") pod \"84eff151-b358-4a70-b27c-4c52f6b9d381\" (UID: \"84eff151-b358-4a70-b27c-4c52f6b9d381\") "
Nov 04 03:09:47.970534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.970504    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca" pod "84eff151-b358-4a70-b27c-4c52f6b9d381" (UID: "84eff151-b358-4a70-b27c-4c52f6b9d381"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:09:47.970946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.970922    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84eff151-b358-4a70-b27c-4c52f6b9d381-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:47.972931 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.972904    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls" pod "84eff151-b358-4a70-b27c-4c52f6b9d381" (UID: "84eff151-b358-4a70-b27c-4c52f6b9d381"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:09:47.972931 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.972923    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk" pod "84eff151-b358-4a70-b27c-4c52f6b9d381" (UID: "84eff151-b358-4a70-b27c-4c52f6b9d381"). InnerVolumeSpecName "kube-api-access-dw7kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:09:47.973197 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:47.973172    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs" pod "84eff151-b358-4a70-b27c-4c52f6b9d381" (UID: "84eff151-b358-4a70-b27c-4c52f6b9d381"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:09:48.072782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.072756    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:48.072782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.072781    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84eff151-b358-4a70-b27c-4c52f6b9d381-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:48.073225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.072797    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw7kk\" (UniqueName: \"kubernetes.io/projected/84eff151-b358-4a70-b27c-4c52f6b9d381-kube-api-access-dw7kk\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:09:48.394577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.394540    6019 generic.go:356] "Generic (PLEG): container finished" podID="84eff151-b358-4a70-b27c-4c52f6b9d381" containerID="d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b" exitCode=0
Nov 04 03:09:48.394769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.394574    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"
Nov 04 03:09:48.394769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.394622    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" event={"ID":"84eff151-b358-4a70-b27c-4c52f6b9d381","Type":"ContainerDied","Data":"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"}
Nov 04 03:09:48.394769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.394691    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg" event={"ID":"84eff151-b358-4a70-b27c-4c52f6b9d381","Type":"ContainerDied","Data":"897974601b958f50c07ba5a09c8d4ba63d2dafe2a2fefc529a0fb7ce4690a8f6"}
Nov 04 03:09:48.394769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.394720    6019 scope.go:122] "RemoveContainer" containerID="d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"
Nov 04 03:09:48.396580 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.396555    6019 generic.go:356] "Generic (PLEG): container finished" podID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerID="30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10" exitCode=0
Nov 04 03:09:48.396693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.396602    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" event={"ID":"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b","Type":"ContainerDied","Data":"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"}
Nov 04 03:09:48.396693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.396617    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"
Nov 04 03:09:48.396693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.396653    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg" event={"ID":"6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b","Type":"ContainerDied","Data":"86cb8c5c60b3c335d0588804f631d5bfdb2154f77a9d156161caafdbafe490c0"}
Nov 04 03:09:48.407156 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.407135    6019 scope.go:122] "RemoveContainer" containerID="d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"
Nov 04 03:09:48.407471 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:48.407451    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b\": container with ID starting with d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b not found: ID does not exist" containerID="d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"
Nov 04 03:09:48.407553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.407480    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b"} err="failed to get container status \"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b\": rpc error: code = NotFound desc = could not find container \"d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b\": container with ID starting with d9e18e6f63282217795436365bf7713df0bde0dd4a62485de9c0e8687288525b not found: ID does not exist"
Nov 04 03:09:48.407553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.407496    6019 scope.go:122] "RemoveContainer" containerID="30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"
Nov 04 03:09:48.416973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.416953    6019 scope.go:122] "RemoveContainer" containerID="30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"
Nov 04 03:09:48.417265 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:09:48.417246    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10\": container with ID starting with 30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10 not found: ID does not exist" containerID="30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"
Nov 04 03:09:48.417346 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.417273    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10"} err="failed to get container status \"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10\": rpc error: code = NotFound desc = could not find container \"30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10\": container with ID starting with 30f6b954b1b575192315b0ea8654c78309d928cc4bac04af60f52117dd5c1b10 not found: ID does not exist"
Nov 04 03:09:48.560270 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.560236    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:09:48.607994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.607913    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569fb7-88c8b6848-fbthg"]
Nov 04 03:09:48.609398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.609368    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:48.728669 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.728635    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 03:09:48.737150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.736671    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg"]
Nov 04 03:09:48.802596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.802560    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:48.957095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.956952    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:48.983416 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.983391    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" path="/var/lib/kubelet/pods/6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b/volumes"
Nov 04 03:09:48.983927 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:48.983910    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84eff151-b358-4a70-b27c-4c52f6b9d381" path="/var/lib/kubelet/pods/84eff151-b358-4a70-b27c-4c52f6b9d381/volumes"
Nov 04 03:09:49.117692 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:49.117664    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:51.164982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:51.164946    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:51.702637 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:51.702597    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:52.497837 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:52.497806    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:53.339462 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:53.339418    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:53.482562 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:53.482529    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:53.709880 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:53.709777    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:54.655713 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:54.655627    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.140123 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.140089    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.285354 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.285321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.517021 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.516986    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.667004 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.666974    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.825297 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.825217    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:55.976471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:55.976411    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:56.228832 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:56.228749    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:56.826205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:56.826169    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:57.428457 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.428427    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:09:57.429113 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429094    6019 cpu_manager.go:414] "containerMap: removing container" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429114    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429138    6019 cpu_manager.go:414] "containerMap: removing container" podUID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" containerName="payload-pre-processing"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429144    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" containerName="payload-pre-processing"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429157    6019 cpu_manager.go:414] "containerMap: removing container" podUID="84eff151-b358-4a70-b27c-4c52f6b9d381" containerName="maas-api"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429163    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84eff151-b358-4a70-b27c-4c52f6b9d381" containerName="maas-api"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429170    6019 cpu_manager.go:414] "containerMap: removing container" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429174    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429186    6019 cpu_manager.go:414] "containerMap: removing container" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429190    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429198    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" containerName="maas-api"
Nov 04 03:09:57.429205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429203    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" containerName="maas-api"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429216    6019 cpu_manager.go:414] "containerMap: removing container" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429220    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429236    6019 cpu_manager.go:414] "containerMap: removing container" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429240    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429247    6019 cpu_manager.go:414] "containerMap: removing container" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429251    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429259    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429263    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429276    6019 cpu_manager.go:414] "containerMap: removing container" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429280    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429372    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6288f05-af04-46c0-bb21-71a943669694" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429383    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb95735c-4dea-4c41-adbf-2fddd9288f2b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429389    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="769a6f5a-1811-4106-a8a7-b1636a39218b" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429396    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e6da0ef-ea2f-46e1-ad1f-5cd663cb028b" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429403    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429409    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="b70d340e-cc4c-4e8d-8c9d-f4660894d103" containerName="main"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429413    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429419    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429427    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="0656dde6-e52a-4fce-89d1-aa4fcd4055fe" containerName="payload-pre-processing"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429433    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="84eff151-b358-4a70-b27c-4c52f6b9d381" containerName="maas-api"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429439    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429445    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="76d45294-a924-42cf-8e63-c31279716771" containerName="payload-processing"
Nov 04 03:09:57.429597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.429452    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a3ab4ae-9eb9-44df-ac53-6f8d0b8e0fa3" containerName="maas-api"
Nov 04 03:09:57.432833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.432815    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.478174 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.478150    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:09:57.478742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.478724    6019 cpu_manager.go:414] "containerMap: removing container" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.478790 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.478746    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d6dbeb7-e0be-45b7-9528-1b84c8dd1fb1" containerName="istio-proxy"
Nov 04 03:09:57.478825 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.478791    6019 cpu_manager.go:414] "containerMap: removing container" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.478825 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.478796    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce29f7ec-27de-48f2-9479-c7e0d4712964" containerName="istio-proxy"
Nov 04 03:09:57.482191 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.482171    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.533188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.533158    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:09:57.556299 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556264    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556299 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556298    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.556517 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556317    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdfl9\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556517 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556419    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556517 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556441    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.556517 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556467    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556517 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556486    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556598    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556631    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.556770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556653    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.556770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556697    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.556770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556736    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.557010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556796    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.557010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556836    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.557010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.556876    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.557224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.557008    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.557224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.557038    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lk5g\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.557224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.557106    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.594432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.594396    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:09:57.657994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.657962    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.657994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.657995    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658014    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658100    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.658291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658136    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.658291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658203    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658523 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658320    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658523 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658436    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lk5g\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658523 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658487    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658523 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658492    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658547    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658575    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658574    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658601    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658622    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.658742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658624    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdfl9\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658785    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658821    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658827    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.658875    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659030    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659124    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659132    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659161    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.659673 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659263    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.659673 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659511    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.659673 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659518    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.659834 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.659689    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.660782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.660751    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.661467 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.661445    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.661577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.661561    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.662444 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.662422    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.719403 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.719333    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.719555 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.719443    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.719555 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.719467    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdfl9\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.721448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.721429    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lk5g\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\") pod \"maas-default-gateway-openshift-default-76597d6855-6cfhw\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:57.746354 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.746324    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:57.799111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:57.796990    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:58.222359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.217572    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:09:58.236695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.235694    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.236695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.235787    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.236695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.235828    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.264223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.264180    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:09:58.267341 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:09:58.267310    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a5b1fb_c31a_49e1_a4bb_9f7492a9e73e.slice/crio-48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b WatchSource:0}: Error finding container 48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b: Status 404 returned error can't find the container with id 48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b
Nov 04 03:09:58.270392 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.270361    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.270494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.270428    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.270494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.270457    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:09:58.470105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.469999    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" event={"ID":"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e","Type":"ContainerStarted","Data":"4ed1a2c2ba843f60b97629cf0f6d26e8043189304a283bbb0842dbd145924060"}
Nov 04 03:09:58.470105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.470052    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" event={"ID":"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e","Type":"ContainerStarted","Data":"48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b"}
Nov 04 03:09:58.471922 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.471894    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" event={"ID":"b5201791-56c8-4259-aed8-0c9cdb4197a2","Type":"ContainerStarted","Data":"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"}
Nov 04 03:09:58.472133 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.471946    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" event={"ID":"b5201791-56c8-4259-aed8-0c9cdb4197a2","Type":"ContainerStarted","Data":"f507bf97c28b9a97c9fc99d499d5f4f31eab1a36909ab1abe81e495df9b90944"}
Nov 04 03:09:58.597114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.597034    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podStartSLOduration=1.59701732 podStartE2EDuration="1.59701732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:09:58.594282078 +0000 UTC m=+2153.735295487" watchObservedRunningTime="2026-11-04 03:09:58.59701732 +0000 UTC m=+2153.738030728"
Nov 04 03:09:58.718305 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.718245    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podStartSLOduration=1.718229958 podStartE2EDuration="1.718229958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:09:58.717339362 +0000 UTC m=+2153.858352770" watchObservedRunningTime="2026-11-04 03:09:58.718229958 +0000 UTC m=+2153.859243365"
Nov 04 03:09:58.748116 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.747973    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:09:58.801230 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:58.801196    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:09:59.518383 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:59.518346    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:09:59.771152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:59.756210    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:09:59.771152 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:59.756282    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:09:59.800038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:59.800001    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:09:59.800318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:09:59.800291    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:00.143988 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.142568    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:00.463157 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.442925    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:00.747771 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.747689    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:00.747771 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.747750    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:00.798062 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.798019    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:00.798399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:00.798369    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:01.209614 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:01.209581    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:01.747534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:01.747499    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:01.747718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:01.747556    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:01.798635 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:01.798587    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:01.799048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:01.798657    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:02.746757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:02.746722    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:02.746950 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:02.746780    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:02.798546 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:02.798501    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:02.798734 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:02.798575    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:03.495097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.494891    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:03.647130 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.647066    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:03.747437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.747341    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:03.747437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.747409    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:03.798592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.798552    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:03.798864 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:03.798838    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:04.220253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:04.220217    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:04.747115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:04.747050    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:04.747301 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:04.747150    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:04.798008 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:04.797965    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:04.798234 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:04.798027    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:05.093954 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.093913    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:05.269632 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.269592    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:05.483592 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.483491    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:05.747045 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.746935    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 04 03:10:05.747045 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.747005    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 04 03:10:05.798700 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.798652    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:05.798898 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:05.798732    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:06.107227 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.107186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:06.674792 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:06.674724    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:10:06.751449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.751405    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:10:06.751931 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.751910    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:10:06.752767 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.752747    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:10:06.797981 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.797939    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-6cfhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:10:06.798225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.798003    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:10:06.833709 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:06.833666    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:07.834536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:07.834492    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:10:07.834984 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:07.834870    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:10:07.836064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:07.836036    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:10:10.243616 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:10.243579    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:11.048176 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.048120    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:11.175506 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.175472    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:10:11.175713 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.175671    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" podUID="a8bb3939-e67b-4b37-91ce-d384f2021f95" containerName="maas-api" containerID="cri-o://5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38" gracePeriod=30
Nov 04 03:10:11.249130 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.249051    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:11.425110 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.423771    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:11.553360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.553329    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:10:11.659169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.658588    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\") pod \"a8bb3939-e67b-4b37-91ce-d384f2021f95\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") "
Nov 04 03:10:11.659169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.658794    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\" (UniqueName: \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\") pod \"a8bb3939-e67b-4b37-91ce-d384f2021f95\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") "
Nov 04 03:10:11.659169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.658868    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") pod \"a8bb3939-e67b-4b37-91ce-d384f2021f95\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") "
Nov 04 03:10:11.659169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.658908    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\") pod \"a8bb3939-e67b-4b37-91ce-d384f2021f95\" (UID: \"a8bb3939-e67b-4b37-91ce-d384f2021f95\") "
Nov 04 03:10:11.662100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.660473    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca" pod "a8bb3939-e67b-4b37-91ce-d384f2021f95" (UID: "a8bb3939-e67b-4b37-91ce-d384f2021f95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:10:11.666092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.665231    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs" pod "a8bb3939-e67b-4b37-91ce-d384f2021f95" (UID: "a8bb3939-e67b-4b37-91ce-d384f2021f95"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:10:11.666092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.665375    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls" pod "a8bb3939-e67b-4b37-91ce-d384f2021f95" (UID: "a8bb3939-e67b-4b37-91ce-d384f2021f95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:10:11.666092 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.665619    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq" pod "a8bb3939-e67b-4b37-91ce-d384f2021f95" (UID: "a8bb3939-e67b-4b37-91ce-d384f2021f95"). InnerVolumeSpecName "kube-api-access-hs4tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:11.733100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.728964    6019 generic.go:356] "Generic (PLEG): container finished" podID="a8bb3939-e67b-4b37-91ce-d384f2021f95" containerID="5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38" exitCode=0
Nov 04 03:10:11.733100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.729058    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" event={"ID":"a8bb3939-e67b-4b37-91ce-d384f2021f95","Type":"ContainerDied","Data":"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"}
Nov 04 03:10:11.733100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.729134    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99" event={"ID":"a8bb3939-e67b-4b37-91ce-d384f2021f95","Type":"ContainerDied","Data":"57c950459094150f6e80a65d4e6d11c59bbc1f211088ea526e16f83759f9b6e6"}
Nov 04 03:10:11.733100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.729166    6019 scope.go:122] "RemoveContainer" containerID="5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"
Nov 04 03:10:11.733100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.730685    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"
Nov 04 03:10:11.762534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.760439    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs4tq\" (UniqueName: \"kubernetes.io/projected/a8bb3939-e67b-4b37-91ce-d384f2021f95-kube-api-access-hs4tq\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:11.762534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.760476    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:11.762534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.760491    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8bb3939-e67b-4b37-91ce-d384f2021f95-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:11.762534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.760506    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bb3939-e67b-4b37-91ce-d384f2021f95-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:11.807297 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.803430    6019 scope.go:122] "RemoveContainer" containerID="5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"
Nov 04 03:10:11.807297 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:11.803987    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38\": container with ID starting with 5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38 not found: ID does not exist" containerID="5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"
Nov 04 03:10:11.807297 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.804028    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38"} err="failed to get container status \"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38\": rpc error: code = NotFound desc = could not find container \"5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38\": container with ID starting with 5e2c0a853e05471eec4b3ea964de708b13a22a824ff5b320929daa833c99be38 not found: ID does not exist"
Nov 04 03:10:11.817414 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.817240    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:11.912019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.911934    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:10:11.915920 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.915892    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-00c846-fdf4fbbb8-pdl99"]
Nov 04 03:10:11.962760 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:11.962730    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:12.362535 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.362497    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 03:10:12.363024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.362702    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" podUID="70029146-61e2-4890-a9f9-583ef339a4c9" containerName="maas-api" containerID="cri-o://f71b316d956971e61a2e279fc11feff4fb46fbbea75816c2ede5e137a6c57417" gracePeriod=30
Nov 04 03:10:12.475446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.475402    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:12.565027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.558767    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:12.565027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.559432    6019 cpu_manager.go:414] "containerMap: removing container" podUID="a8bb3939-e67b-4b37-91ce-d384f2021f95" containerName="maas-api"
Nov 04 03:10:12.565027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.559448    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8bb3939-e67b-4b37-91ce-d384f2021f95" containerName="maas-api"
Nov 04 03:10:12.565027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.559544    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8bb3939-e67b-4b37-91ce-d384f2021f95" containerName="maas-api"
Nov 04 03:10:12.637304 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.637203    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.697669 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.697640    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:12.717680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717647    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.717883 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717713    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.717883 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717756    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ldx\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.717883 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717785    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.717883 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717833    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.718117 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.717941    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.718117 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.718014    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.718232 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.718116    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.718232 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.718191    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.740873 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.740835    6019 generic.go:356] "Generic (PLEG): container finished" podID="70029146-61e2-4890-a9f9-583ef339a4c9" containerID="f71b316d956971e61a2e279fc11feff4fb46fbbea75816c2ede5e137a6c57417" exitCode=0
Nov 04 03:10:12.741036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.740972    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" event={"ID":"70029146-61e2-4890-a9f9-583ef339a4c9","Type":"ContainerDied","Data":"f71b316d956971e61a2e279fc11feff4fb46fbbea75816c2ede5e137a6c57417"}
Nov 04 03:10:12.741036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.741025    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq" event={"ID":"70029146-61e2-4890-a9f9-583ef339a4c9","Type":"ContainerDied","Data":"67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174"}
Nov 04 03:10:12.741135 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.741049    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67d3a4e687a3490038522dda61b0a6b0b323f807d39400676da5f7f8592ee174"
Nov 04 03:10:12.801109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.779485    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916578    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916675    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916747    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916806    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ldx\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916855    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.916914    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.917043    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.917148    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.917221    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.919860    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.920843    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.921531    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.922276    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.927140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.926410    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.938230 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.938191    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:12.939348 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.939261    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.941340 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.941308    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.980647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.980612    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.980821 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.980707    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ldx\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\") pod \"e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") " pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:12.982870 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:12.982842    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8bb3939-e67b-4b37-91ce-d384f2021f95" path="/var/lib/kubelet/pods/a8bb3939-e67b-4b37-91ce-d384f2021f95/volumes"
Nov 04 03:10:13.017561 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.017520    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:13.018597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.017935    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\") pod \"70029146-61e2-4890-a9f9-583ef339a4c9\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") "
Nov 04 03:10:13.018597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.018247    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\" (UniqueName: \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\") pod \"70029146-61e2-4890-a9f9-583ef339a4c9\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") "
Nov 04 03:10:13.018597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.018357    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") pod \"70029146-61e2-4890-a9f9-583ef339a4c9\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") "
Nov 04 03:10:13.018597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.018491    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\") pod \"70029146-61e2-4890-a9f9-583ef339a4c9\" (UID: \"70029146-61e2-4890-a9f9-583ef339a4c9\") "
Nov 04 03:10:13.019377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.019333    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca" pod "70029146-61e2-4890-a9f9-583ef339a4c9" (UID: "70029146-61e2-4890-a9f9-583ef339a4c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:10:13.021114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.021018    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs" pod "70029146-61e2-4890-a9f9-583ef339a4c9" (UID: "70029146-61e2-4890-a9f9-583ef339a4c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:10:13.021341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.021305    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls" pod "70029146-61e2-4890-a9f9-583ef339a4c9" (UID: "70029146-61e2-4890-a9f9-583ef339a4c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:10:13.021569 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.021548    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25" pod "70029146-61e2-4890-a9f9-583ef339a4c9" (UID: "70029146-61e2-4890-a9f9-583ef339a4c9"). InnerVolumeSpecName "kube-api-access-tps25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:13.120851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.120819    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:13.120851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.120850    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70029146-61e2-4890-a9f9-583ef339a4c9-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:13.120851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.120862    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70029146-61e2-4890-a9f9-583ef339a4c9-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:13.121118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.120871    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tps25\" (UniqueName: \"kubernetes.io/projected/70029146-61e2-4890-a9f9-583ef339a4c9-kube-api-access-tps25\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:13.504827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.504788    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:13.515012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.514962    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:10:13.515219 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.515063    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:10:13.515219 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.515199    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:10:13.751667 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.751628    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" event={"ID":"439f449b-6da3-4df3-9c10-533714b438f1","Type":"ContainerStarted","Data":"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"}
Nov 04 03:10:13.751667 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.751663    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"
Nov 04 03:10:13.751889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.751698    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" event={"ID":"439f449b-6da3-4df3-9c10-533714b438f1","Type":"ContainerStarted","Data":"57efec8f474f8841758fbcbfb46a9e505be71169ceb8cf18fa46877a9a66e53a"}
Nov 04 03:10:13.875192 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:13.874726    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podStartSLOduration=1.8747126029999999 podStartE2EDuration="1.874712603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:10:13.871464688 +0000 UTC m=+2169.012478123" watchObservedRunningTime="2026-11-04 03:10:13.874712603 +0000 UTC m=+2169.015726009"
Nov 04 03:10:14.018407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.018311    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:14.019911 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.019881    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:14.020058 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.019934    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:14.037927 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.037892    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 03:10:14.041888 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.041855    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b7476-54fd66b4f-tv4vq"]
Nov 04 03:10:14.275446 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.275353    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:14.983892 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:14.983861    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70029146-61e2-4890-a9f9-583ef339a4c9" path="/var/lib/kubelet/pods/70029146-61e2-4890-a9f9-583ef339a4c9/volumes"
Nov 04 03:10:15.018203 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:15.018170    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:15.018407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:15.018229    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:15.873839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:15.873792    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:16.017918 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:16.017880    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:16.018360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:16.017942    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:16.018434 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:16.018364    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:17.018539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:17.018508    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:17.018906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:17.018555    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:17.574110 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:17.574027    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:18.018223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:18.018186    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:18.018384 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:18.018244    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:18.090389 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:18.090351    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:19.018128 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.018088    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:19.018317 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.018143    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:19.074042 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.073997    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:19.217266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.217230    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:19.474623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.474525    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:19.965109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:19.949626    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:20.042102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:20.039501    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:20.042102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:20.039572    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:20.261751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:20.261675    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:21.018001 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:21.017964    6019 patch_prober.go:29] interesting pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 04 03:10:21.018224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:21.018019    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 04 03:10:22.022562 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:22.022530    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:22.022968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:22.022935    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:22.023546 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:22.023529    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:24.320337 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:24.320296    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:24.482158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:24.482120    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:24.648471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:24.648389    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:24.924277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:24.924190    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:26.040418 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.040362    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:26.534056 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.534009    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:26.538056 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.538029    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:26.538242 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.538217    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy" containerID="cri-o://1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04" gracePeriod=30
Nov 04 03:10:26.540482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.540461    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:10:26.540644 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.540626    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy" containerID="cri-o://58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8" gracePeriod=30
Nov 04 03:10:26.562839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.562750    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 03:10:26.563113 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:26.562959    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy" containerID="cri-o://c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f" gracePeriod=30
Nov 04 03:10:28.275817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:28.275774    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:28.510503 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:28.510458    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:30.297447 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:30.297411    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:31.832041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.832012    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:10:31.860410 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.860379    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:31.865345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865305    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.865489 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865370    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.865564 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865498    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.865621 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865572    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.865711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865693    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.865774 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865748    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865826    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865866    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865901    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\") pod \"b5201791-56c8-4259-aed8-0c9cdb4197a2\" (UID: \"b5201791-56c8-4259-aed8-0c9cdb4197a2\") "
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.865956    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866256    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866289    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866472    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866588    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866715    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866739    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866755    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5201791-56c8-4259-aed8-0c9cdb4197a2-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866768    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.866804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.866782    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.869100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.869055    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "kube-api-access-wdfl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.870439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.870417    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.870519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.870451    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.870519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.870447    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo" pod "b5201791-56c8-4259-aed8-0c9cdb4197a2" (UID: "b5201791-56c8-4259-aed8-0c9cdb4197a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:10:31.907253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.907228    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 03:10:31.909202 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.909176    6019 generic.go:356] "Generic (PLEG): container finished" podID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerID="c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f" exitCode=0
Nov 04 03:10:31.909318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.909214    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"
Nov 04 03:10:31.909318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.909257    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" event={"ID":"bfa9edf1-0bca-4cc2-b8db-2420bedcad59","Type":"ContainerDied","Data":"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"}
Nov 04 03:10:31.909412 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.909335    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5" event={"ID":"bfa9edf1-0bca-4cc2-b8db-2420bedcad59","Type":"ContainerDied","Data":"048a85463eb1b6a5bb25b4974d2841b05ee7205665bef519eb6cc64d6499c7c9"}
Nov 04 03:10:31.909412 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.909361    6019 scope.go:122] "RemoveContainer" containerID="c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"
Nov 04 03:10:31.910643 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.910624    6019 generic.go:356] "Generic (PLEG): container finished" podID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerID="58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8" exitCode=0
Nov 04 03:10:31.910722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.910676    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"
Nov 04 03:10:31.910722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.910701    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" event={"ID":"b5201791-56c8-4259-aed8-0c9cdb4197a2","Type":"ContainerDied","Data":"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"}
Nov 04 03:10:31.910808 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.910738    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56" event={"ID":"b5201791-56c8-4259-aed8-0c9cdb4197a2","Type":"ContainerDied","Data":"f507bf97c28b9a97c9fc99d499d5f4f31eab1a36909ab1abe81e495df9b90944"}
Nov 04 03:10:31.911927 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.911908    6019 generic.go:356] "Generic (PLEG): container finished" podID="439f449b-6da3-4df3-9c10-533714b438f1" containerID="1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04" exitCode=0
Nov 04 03:10:31.912014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.911943    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"
Nov 04 03:10:31.912014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.911978    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" event={"ID":"439f449b-6da3-4df3-9c10-533714b438f1","Type":"ContainerDied","Data":"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"}
Nov 04 03:10:31.912014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.912007    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp" event={"ID":"439f449b-6da3-4df3-9c10-533714b438f1","Type":"ContainerDied","Data":"57efec8f474f8841758fbcbfb46a9e505be71169ceb8cf18fa46877a9a66e53a"}
Nov 04 03:10:31.920167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.920149    6019 scope.go:122] "RemoveContainer" containerID="c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"
Nov 04 03:10:31.920422 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:31.920406    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f\": container with ID starting with c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f not found: ID does not exist" containerID="c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"
Nov 04 03:10:31.920480 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.920430    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f"} err="failed to get container status \"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f\": rpc error: code = NotFound desc = could not find container \"c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f\": container with ID starting with c39e19fea4860be7c470bc07944264437fb4a0f14543750afdce7dd998a5271f not found: ID does not exist"
Nov 04 03:10:31.920480 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.920452    6019 scope.go:122] "RemoveContainer" containerID="58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"
Nov 04 03:10:31.929541 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.929524    6019 scope.go:122] "RemoveContainer" containerID="58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"
Nov 04 03:10:31.929768 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:31.929748    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8\": container with ID starting with 58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8 not found: ID does not exist" containerID="58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"
Nov 04 03:10:31.929818 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.929775    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8"} err="failed to get container status \"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8\": rpc error: code = NotFound desc = could not find container \"58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8\": container with ID starting with 58e68203358d449ade1861e4e63f91d31fc1450b26f233fb2b28f9734d5f28a8 not found: ID does not exist"
Nov 04 03:10:31.929818 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.929791    6019 scope.go:122] "RemoveContainer" containerID="1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"
Nov 04 03:10:31.938316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.938301    6019 scope.go:122] "RemoveContainer" containerID="1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"
Nov 04 03:10:31.938579 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:31.938561    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04\": container with ID starting with 1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04 not found: ID does not exist" containerID="1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"
Nov 04 03:10:31.938634 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.938585    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04"} err="failed to get container status \"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04\": rpc error: code = NotFound desc = could not find container \"1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04\": container with ID starting with 1b30f29abe5cb75b36a653ab3e2706d3bc0c05165f8de2065526052074254a04 not found: ID does not exist"
Nov 04 03:10:31.967416 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967398    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967439    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967458    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.967505 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967487    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.967679 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967513    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967679 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967530    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.967679 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967576    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.967679 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967625    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967722    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967701    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.967875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967757    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.967875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967783    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.967875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967788    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967936    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967935    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967869    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.967998    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968035    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968064    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.968166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968123    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.968626 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968199    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\") pod \"439f449b-6da3-4df3-9c10-533714b438f1\" (UID: \"439f449b-6da3-4df3-9c10-533714b438f1\") "
Nov 04 03:10:31.968626 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968314    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.968724 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968625    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\") pod \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\" (UID: \"bfa9edf1-0bca-4cc2-b8db-2420bedcad59\") "
Nov 04 03:10:31.968724 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968605    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.968991 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.968970    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.969166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969137    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.969242 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969192    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:10:31.969297 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969269    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.969360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969292    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969314    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969331    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969345    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969358    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969371    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969385    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969398    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969412    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdfl9\" (UniqueName: \"kubernetes.io/projected/b5201791-56c8-4259-aed8-0c9cdb4197a2-kube-api-access-wdfl9\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969429    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5201791-56c8-4259-aed8-0c9cdb4197a2-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969443    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.969633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.969457    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:31.970277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.970249    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.970840 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.970810    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "kube-api-access-6slbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.971144 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.971123    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:10:31.971300 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.971276    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:10:31.971384 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.971296    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token" pod "bfa9edf1-0bca-4cc2-b8db-2420bedcad59" (UID: "bfa9edf1-0bca-4cc2-b8db-2420bedcad59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.971442 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.971423    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "kube-api-access-q9ldx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.971484 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.971421    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:10:31.972256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:31.972239    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo" pod "439f449b-6da3-4df3-9c10-533714b438f1" (UID: "439f449b-6da3-4df3-9c10-533714b438f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:10:32.071207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071122    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439f449b-6da3-4df3-9c10-533714b438f1-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071159    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071175    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6slbt\" (UniqueName: \"kubernetes.io/projected/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-kube-api-access-6slbt\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071190    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9ldx\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-kube-api-access-q9ldx\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071207 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071203    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071219    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439f449b-6da3-4df3-9c10-533714b438f1-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071233    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439f449b-6da3-4df3-9c10-533714b438f1-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071246    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071260    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfa9edf1-0bca-4cc2-b8db-2420bedcad59-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.071492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.071272    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439f449b-6da3-4df3-9c10-533714b438f1-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:10:32.355057 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.354972    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:10:32.359246 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.359219    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56"]
Nov 04 03:10:32.466664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.466622    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:32.471321 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.471293    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp"]
Nov 04 03:10:32.581658 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.581628    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 03:10:32.586659 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.586638    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5"]
Nov 04 03:10:32.982667 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.982633    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="439f449b-6da3-4df3-9c10-533714b438f1" path="/var/lib/kubelet/pods/439f449b-6da3-4df3-9c10-533714b438f1/volumes"
Nov 04 03:10:32.983198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.983182    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" path="/var/lib/kubelet/pods/b5201791-56c8-4259-aed8-0c9cdb4197a2/volumes"
Nov 04 03:10:32.983619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:32.983605    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" path="/var/lib/kubelet/pods/bfa9edf1-0bca-4cc2-b8db-2420bedcad59/volumes"
Nov 04 03:10:34.669534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:34.669494    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:36.524864 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:36.524830    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:36.646419 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:10:36.646383    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:10:37.423408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:37.423375    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:37.589791 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:37.589750    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:38.216268 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:38.216230    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:38.427023 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:38.426976    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:39.155372 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:39.155331    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:39.737466 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:39.737431    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:40.348427 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:40.348392    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:41.818232 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:41.818196    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:41.973753 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:41.967118    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:44.116566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:44.116516    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:46.903764 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:46.903725    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:47.100707 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:47.100674    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:49.079330 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:49.079297    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 03:10:49.128795 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:49.128761    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 03:10:49.242932 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:49.242895    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:49.914439 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:49.914362    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:50.073229 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:50.073198    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:52.147117 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:52.147034    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:53.120496 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:53.120457    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:55.350757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:55.350719    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:56.483480 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:56.483449    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:58.292409 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:58.292374    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:10:59.582060 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:10:59.582002    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:00.615150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:00.615117    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:00.759301 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:00.759267    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:02.749310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:02.749273    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:11:02.749807 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:02.749528    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" podUID="e7fdc64a-59ac-4958-a908-fa164927652f" containerName="maas-api" containerID="cri-o://915a8af52f0c74ab578b299f86c26c12848d5edacfb598d4ce9a78802175c1e4" gracePeriod=30
Nov 04 03:11:02.864673 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:02.864642    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:03.240199 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.226041    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:03.286458 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.285517    6019 generic.go:356] "Generic (PLEG): container finished" podID="e7fdc64a-59ac-4958-a908-fa164927652f" containerID="915a8af52f0c74ab578b299f86c26c12848d5edacfb598d4ce9a78802175c1e4" exitCode=0
Nov 04 03:11:03.286458 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.285638    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" event={"ID":"e7fdc64a-59ac-4958-a908-fa164927652f","Type":"ContainerDied","Data":"915a8af52f0c74ab578b299f86c26c12848d5edacfb598d4ce9a78802175c1e4"}
Nov 04 03:11:03.286458 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.285688    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf" event={"ID":"e7fdc64a-59ac-4958-a908-fa164927652f","Type":"ContainerDied","Data":"2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1"}
Nov 04 03:11:03.289420 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.285746    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2557a276f1d2da6778e4c86d16c10507ac28ff40613a0ff6d993b81d7cb76cc1"
Nov 04 03:11:03.290971 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.290950    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:11:03.405196 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.398686    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:03.434898 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.434860    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\") pod \"e7fdc64a-59ac-4958-a908-fa164927652f\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") "
Nov 04 03:11:03.435385 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.435346    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") pod \"e7fdc64a-59ac-4958-a908-fa164927652f\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") "
Nov 04 03:11:03.435653 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.435628    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\" (UniqueName: \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\") pod \"e7fdc64a-59ac-4958-a908-fa164927652f\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") "
Nov 04 03:11:03.435959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.435940    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\") pod \"e7fdc64a-59ac-4958-a908-fa164927652f\" (UID: \"e7fdc64a-59ac-4958-a908-fa164927652f\") "
Nov 04 03:11:03.445296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.445261    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca" pod "e7fdc64a-59ac-4958-a908-fa164927652f" (UID: "e7fdc64a-59ac-4958-a908-fa164927652f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:03.487098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.480227    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs" pod "e7fdc64a-59ac-4958-a908-fa164927652f" (UID: "e7fdc64a-59ac-4958-a908-fa164927652f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:03.541905 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.537840    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fdc64a-59ac-4958-a908-fa164927652f-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:03.541905 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.537891    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:03.566097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.561317    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls" pod "e7fdc64a-59ac-4958-a908-fa164927652f" (UID: "e7fdc64a-59ac-4958-a908-fa164927652f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:03.573065 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.572959    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:03.634095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.629437    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj" pod "e7fdc64a-59ac-4958-a908-fa164927652f" (UID: "e7fdc64a-59ac-4958-a908-fa164927652f"). InnerVolumeSpecName "kube-api-access-kpbwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:03.639880 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.639836    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fdc64a-59ac-4958-a908-fa164927652f-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:03.640261 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.640055    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpbwj\" (UniqueName: \"kubernetes.io/projected/e7fdc64a-59ac-4958-a908-fa164927652f-kube-api-access-kpbwj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:03.777685 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:03.777646    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:04.136463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.136408    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" containerID="cri-o://8fab2b144f349f3eba7836a46921bda4d7dc4f3bcfdfc8bd6e5bc3af5ec1725a" gracePeriod=45
Nov 04 03:11:04.185694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.185625    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" containerID="cri-o://0a82e35d3a56b532fbeac9698f8c814e0ec123760953de7b426cf407a836bd21" gracePeriod=45
Nov 04 03:11:04.338107 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.334720    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"
Nov 04 03:11:04.384449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.384414    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:04.487767 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.484589    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": dial tcp 10.132.0.47:8000: connect: connection refused"
Nov 04 03:11:04.543713 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.543555    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:04.593761 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.593722    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:11:04.643099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.640194    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-22e72e-59b8b8f4c9-ckpqf"]
Nov 04 03:11:04.941143 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.940613    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": dial tcp 10.132.0.66:8000: connect: connection refused"
Nov 04 03:11:04.998475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:04.998425    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7fdc64a-59ac-4958-a908-fa164927652f" path="/var/lib/kubelet/pods/e7fdc64a-59ac-4958-a908-fa164927652f/volumes"
Nov 04 03:11:05.453875 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.453828    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:05.554166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.496272    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": dial tcp 10.132.0.47:8000: connect: connection refused"
Nov 04 03:11:05.554166 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.496481    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 03:11:05.731096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.719613    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:05.753180 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.752249    6019 generic.go:356] "Generic (PLEG): container finished" podID="fc095802-ca07-4d29-8d08-0c82006f2541" containerID="8fab2b144f349f3eba7836a46921bda4d7dc4f3bcfdfc8bd6e5bc3af5ec1725a" exitCode=0
Nov 04 03:11:05.753180 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.752476    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" event={"ID":"fc095802-ca07-4d29-8d08-0c82006f2541","Type":"ContainerDied","Data":"8fab2b144f349f3eba7836a46921bda4d7dc4f3bcfdfc8bd6e5bc3af5ec1725a"}
Nov 04 03:11:05.758210 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.757586    6019 generic.go:356] "Generic (PLEG): container finished" podID="09644268-19f7-48c9-9542-f80708a833d8" containerID="0a82e35d3a56b532fbeac9698f8c814e0ec123760953de7b426cf407a836bd21" exitCode=0
Nov 04 03:11:05.758210 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.757787    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" event={"ID":"09644268-19f7-48c9-9542-f80708a833d8","Type":"ContainerDied","Data":"0a82e35d3a56b532fbeac9698f8c814e0ec123760953de7b426cf407a836bd21"}
Nov 04 03:11:05.817382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.817295    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 03:11:05.844109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.840123    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.844109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.840232    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.844109 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.840278    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.844813 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.844776    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:05.855258 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.855226    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.855721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.855701    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.856184 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.856162    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\" (UniqueName: \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\") pod \"fc095802-ca07-4d29-8d08-0c82006f2541\" (UID: \"fc095802-ca07-4d29-8d08-0c82006f2541\") "
Nov 04 03:11:05.857274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.857253    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:05.857743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.855881    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:05.857863 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.857438    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:05.910102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.908736    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:05.926025 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.924241    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:05.947119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.938236    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q" pod "fc095802-ca07-4d29-8d08-0c82006f2541" (UID: "fc095802-ca07-4d29-8d08-0c82006f2541"). InnerVolumeSpecName "kube-api-access-56c6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:05.999097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.998869    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56c6q\" (UniqueName: \"kubernetes.io/projected/fc095802-ca07-4d29-8d08-0c82006f2541-kube-api-access-56c6q\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:05.999097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.998908    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:05.999097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.998923    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:05.999097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.998934    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc095802-ca07-4d29-8d08-0c82006f2541-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:05.999097 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:05.998946    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc095802-ca07-4d29-8d08-0c82006f2541-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.043822 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.043783    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:06.227038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.213230    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:06.364546 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.364517    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 03:11:06.420928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.420896    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.420948    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\" (UniqueName: \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421015    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421114    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421260 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421180    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421260 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421216    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\" (UniqueName: \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\") pod \"09644268-19f7-48c9-9542-f80708a833d8\" (UID: \"09644268-19f7-48c9-9542-f80708a833d8\") "
Nov 04 03:11:06.421374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421287    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:06.421547 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421513    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:06.421652 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421576    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:06.421960 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.421937    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.422147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.422126    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.422260 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.422151    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.423465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.423442    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:06.423575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.423521    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:06.423675 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.423658    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm" pod "09644268-19f7-48c9-9542-f80708a833d8" (UID: "09644268-19f7-48c9-9542-f80708a833d8"). InnerVolumeSpecName "kube-api-access-gtftm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:06.523482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.523447    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09644268-19f7-48c9-9542-f80708a833d8-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.523482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.523479    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtftm\" (UniqueName: \"kubernetes.io/projected/09644268-19f7-48c9-9542-f80708a833d8-kube-api-access-gtftm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.523482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.523490    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09644268-19f7-48c9-9542-f80708a833d8-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:06.599277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.599237    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:06.616762 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:06.616710    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:11:06.767389 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.767358    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"
Nov 04 03:11:06.767602 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.767390    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5" event={"ID":"fc095802-ca07-4d29-8d08-0c82006f2541","Type":"ContainerDied","Data":"dcea90d07194debbe7100018e580c4b11a8c419acc726eb2462796ec271a73f9"}
Nov 04 03:11:06.767602 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.767440    6019 scope.go:122] "RemoveContainer" containerID="8fab2b144f349f3eba7836a46921bda4d7dc4f3bcfdfc8bd6e5bc3af5ec1725a"
Nov 04 03:11:06.775172 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.775104    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q" event={"ID":"09644268-19f7-48c9-9542-f80708a833d8","Type":"ContainerDied","Data":"8d595d58369a4a5d739fd36ce4f94035bd49108b59be59843c4c3c6022dc5626"}
Nov 04 03:11:06.775407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.775381    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"
Nov 04 03:11:06.786786 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.786765    6019 scope.go:122] "RemoveContainer" containerID="0a82e35d3a56b532fbeac9698f8c814e0ec123760953de7b426cf407a836bd21"
Nov 04 03:11:06.960899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:06.960863    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 03:11:07.012367 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.012334    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/simulator-w1-e516ff-kserve-6b5b5ccdf5-88f9q"]
Nov 04 03:11:07.031294 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.031205    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:07.070802 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.070770    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 03:11:07.077401 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.077371    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-e516ff/distinct-2-w1-e516ff-kserve-f56cbdddb-4d7r5"]
Nov 04 03:11:07.215115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.215060    6019 scope.go:122] "RemoveContainer" containerID="f71b316d956971e61a2e279fc11feff4fb46fbbea75816c2ede5e137a6c57417"
Nov 04 03:11:07.224519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.224488    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:07.407839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.407727    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:07.583953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:07.583923    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:08.153320 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:08.142971    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:08.453391 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:08.453251    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:08.981933 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:08.981897    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09644268-19f7-48c9-9542-f80708a833d8" path="/var/lib/kubelet/pods/09644268-19f7-48c9-9542-f80708a833d8/volumes"
Nov 04 03:11:08.982482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:08.982460    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" path="/var/lib/kubelet/pods/fc095802-ca07-4d29-8d08-0c82006f2541/volumes"
Nov 04 03:11:09.619938 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:09.619904    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:09.950440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:09.950356    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:10.600058 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:10.600022    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:10.749043 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:10.749009    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:11.005685 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:11.005652    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:11.559595 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:11.559561    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 03:11:11.608845 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:11.608811    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 03:11:11.713757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:11.713723    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:12.257617 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:12.257570    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:12.592710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:12.592607    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:12.901275 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:12.901187    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:13.052042 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:13.052007    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:13.658585 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:13.658556    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:14.606428 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:14.606393    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 03:11:14.655783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:14.655755    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 03:11:14.747102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:14.747056    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:15.662347 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:15.662312    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 03:11:15.662548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:15.662527    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" podUID="6a59a675-9277-4c29-8c2a-7be0f35f754c" containerName="maas-api" containerID="cri-o://13547f0ac84862898ac0bbf78a777efc500cded4331931ba12e8ca8c17480111" gracePeriod=30
Nov 04 03:11:15.761997 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:15.761949    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:15.835730 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:15.835694    6019 generic.go:356] "Generic (PLEG): container finished" podID="6a59a675-9277-4c29-8c2a-7be0f35f754c" containerID="13547f0ac84862898ac0bbf78a777efc500cded4331931ba12e8ca8c17480111" exitCode=0
Nov 04 03:11:15.835920 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:15.835777    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" event={"ID":"6a59a675-9277-4c29-8c2a-7be0f35f754c","Type":"ContainerDied","Data":"13547f0ac84862898ac0bbf78a777efc500cded4331931ba12e8ca8c17480111"}
Nov 04 03:11:16.021493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.021470    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 03:11:16.076100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.061515    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\") pod \"6a59a675-9277-4c29-8c2a-7be0f35f754c\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") "
Nov 04 03:11:16.076100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.061672    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") pod \"6a59a675-9277-4c29-8c2a-7be0f35f754c\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") "
Nov 04 03:11:16.076100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.061804    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\" (UniqueName: \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\") pod \"6a59a675-9277-4c29-8c2a-7be0f35f754c\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") "
Nov 04 03:11:16.076100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.061858    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\") pod \"6a59a675-9277-4c29-8c2a-7be0f35f754c\" (UID: \"6a59a675-9277-4c29-8c2a-7be0f35f754c\") "
Nov 04 03:11:16.100099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.076778    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca" pod "6a59a675-9277-4c29-8c2a-7be0f35f754c" (UID: "6a59a675-9277-4c29-8c2a-7be0f35f754c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:16.132594 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.132421    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:16.133435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.133382    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls" pod "6a59a675-9277-4c29-8c2a-7be0f35f754c" (UID: "6a59a675-9277-4c29-8c2a-7be0f35f754c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:16.133732 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.133708    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs" pod "6a59a675-9277-4c29-8c2a-7be0f35f754c" (UID: "6a59a675-9277-4c29-8c2a-7be0f35f754c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:16.136440 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.136401    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8" pod "6a59a675-9277-4c29-8c2a-7be0f35f754c" (UID: "6a59a675-9277-4c29-8c2a-7be0f35f754c"). InnerVolumeSpecName "kube-api-access-b7sx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:16.163765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.163729    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59a675-9277-4c29-8c2a-7be0f35f754c-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:16.164553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.164459    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:16.164553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.164493    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a59a675-9277-4c29-8c2a-7be0f35f754c-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:16.164553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.164508    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7sx8\" (UniqueName: \"kubernetes.io/projected/6a59a675-9277-4c29-8c2a-7be0f35f754c-kube-api-access-b7sx8\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:16.843372 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.843339    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"
Nov 04 03:11:16.843851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.843377    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4" event={"ID":"6a59a675-9277-4c29-8c2a-7be0f35f754c","Type":"ContainerDied","Data":"e6b3ccb6b206166a7aa7c0a7b9447a16b9a31d7397de4ecf548f297d12eaaef4"}
Nov 04 03:11:16.843851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:16.843450    6019 scope.go:122] "RemoveContainer" containerID="13547f0ac84862898ac0bbf78a777efc500cded4331931ba12e8ca8c17480111"
Nov 04 03:11:17.007614 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.007573    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 03:11:17.057417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.057387    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e516ff-8446b576c5-2hpf4"]
Nov 04 03:11:17.334616 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.334543    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:11:17.335047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335035    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main"
Nov 04 03:11:17.335112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335049    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main"
Nov 04 03:11:17.335112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335063    6019 cpu_manager.go:414] "containerMap: removing container" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy"
Nov 04 03:11:17.335112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335081    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy"
Nov 04 03:11:17.335112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335098    6019 cpu_manager.go:414] "containerMap: removing container" podUID="6a59a675-9277-4c29-8c2a-7be0f35f754c" containerName="maas-api"
Nov 04 03:11:17.335112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335106    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a59a675-9277-4c29-8c2a-7be0f35f754c" containerName="maas-api"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335118    6019 cpu_manager.go:414] "containerMap: removing container" podUID="70029146-61e2-4890-a9f9-583ef339a4c9" containerName="maas-api"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335123    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70029146-61e2-4890-a9f9-583ef339a4c9" containerName="maas-api"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335133    6019 cpu_manager.go:414] "containerMap: removing container" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335138    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335155    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e7fdc64a-59ac-4958-a908-fa164927652f" containerName="maas-api"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335159    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7fdc64a-59ac-4958-a908-fa164927652f" containerName="maas-api"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335166    6019 cpu_manager.go:414] "containerMap: removing container" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335170    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335178    6019 cpu_manager.go:414] "containerMap: removing container" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335182    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335265    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5201791-56c8-4259-aed8-0c9cdb4197a2" containerName="istio-proxy"
Nov 04 03:11:17.335278 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335278    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7fdc64a-59ac-4958-a908-fa164927652f" containerName="maas-api"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335286    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfa9edf1-0bca-4cc2-b8db-2420bedcad59" containerName="istio-proxy"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335292    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="09644268-19f7-48c9-9542-f80708a833d8" containerName="main"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335301    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc095802-ca07-4d29-8d08-0c82006f2541" containerName="main"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335308    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="439f449b-6da3-4df3-9c10-533714b438f1" containerName="istio-proxy"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335314    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a59a675-9277-4c29-8c2a-7be0f35f754c" containerName="maas-api"
Nov 04 03:11:17.335629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.335318    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="70029146-61e2-4890-a9f9-583ef339a4c9" containerName="maas-api"
Nov 04 03:11:17.339637 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.339620    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.384683 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.384648    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmbm5\" (UniqueName: \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.384683 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.384687    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.384911 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.384821    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.384911 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.384863    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.394916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.394890    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4cbcf738-serving-cert\""
Nov 04 03:11:17.398046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.397915    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:11:17.486555 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.486518    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.486743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.486574    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.486743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.486653    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmbm5\" (UniqueName: \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.486743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.486681    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.486918 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:17.486829    6019 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-serving-cert: secret "maas-api-e2e-mt-4cbcf738-serving-cert" not found
Nov 04 03:11:17.486918 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:17.486901    6019 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls podName:42f270fb-5da0-48c1-9011-64078fb7f5a2 nodeName:}" failed. No retries permitted until 2026-11-04 03:11:17.986884937 +0000 UTC m=+2233.127898337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls") pod "maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" (UID: "42f270fb-5da0-48c1-9011-64078fb7f5a2") : secret "maas-api-e2e-mt-4cbcf738-serving-cert" not found
Nov 04 03:11:17.487217 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.487198    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.489375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.489350    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.548455 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.548421    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmbm5\" (UniqueName: \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.574214 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.574172    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:17.721404 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.721321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:17.867353 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.867317    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:17.993413 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.993335    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:17.995915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:17.995886    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") pod \"maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:18.253260 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:18.253218    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:18.630611 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:18.630513    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:11:18.633698 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:11:18.633670    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f270fb_5da0_48c1_9011_64078fb7f5a2.slice/crio-d3da91e30a89d03b8fd291b346f0c0a541adccd39aa22a1d2cfebcb9e12205d4 WatchSource:0}: Error finding container d3da91e30a89d03b8fd291b346f0c0a541adccd39aa22a1d2cfebcb9e12205d4: Status 404 returned error can't find the container with id d3da91e30a89d03b8fd291b346f0c0a541adccd39aa22a1d2cfebcb9e12205d4
Nov 04 03:11:18.870508 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:18.870471    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" event={"ID":"42f270fb-5da0-48c1-9011-64078fb7f5a2","Type":"ContainerStarted","Data":"d3da91e30a89d03b8fd291b346f0c0a541adccd39aa22a1d2cfebcb9e12205d4"}
Nov 04 03:11:18.981834 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:18.981752    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a59a675-9277-4c29-8c2a-7be0f35f754c" path="/var/lib/kubelet/pods/6a59a675-9277-4c29-8c2a-7be0f35f754c/volumes"
Nov 04 03:11:19.877665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:19.877628    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" event={"ID":"42f270fb-5da0-48c1-9011-64078fb7f5a2","Type":"ContainerStarted","Data":"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"}
Nov 04 03:11:19.878220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:19.877829    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:19.998266 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:19.998202    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" podStartSLOduration=2.143808153 podStartE2EDuration="2.998183619s" totalImagesPullingTime="854.375466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:11:19.994749546 +0000 UTC m=+2235.135762954" watchObservedRunningTime="2026-11-04 03:11:19.998183619 +0000 UTC m=+2235.139197028"
Nov 04 03:11:20.031537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:20.031369    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:21.209346 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:21.209274    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:21.390993 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:21.390942    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:22.393760 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.393728    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:22.476395 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.476355    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 03:11:22.476591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.476570    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" podUID="efe32154-a1cf-4ba0-a775-a95448276eae" containerName="maas-api" containerID="cri-o://ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd" gracePeriod=30
Nov 04 03:11:22.541402 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.541373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:22.686292 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.686252    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:22.851743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.851717    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 03:11:22.897167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.897131    6019 generic.go:356] "Generic (PLEG): container finished" podID="efe32154-a1cf-4ba0-a775-a95448276eae" containerID="ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd" exitCode=0
Nov 04 03:11:22.897311 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.897203    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"
Nov 04 03:11:22.897311 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.897236    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" event={"ID":"efe32154-a1cf-4ba0-a775-a95448276eae","Type":"ContainerDied","Data":"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"}
Nov 04 03:11:22.897311 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.897291    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj" event={"ID":"efe32154-a1cf-4ba0-a775-a95448276eae","Type":"ContainerDied","Data":"7b3bcadc5b9db43ac81aa15be1d5a84ceebb392d2546d5c7b15a2c4c630bf474"}
Nov 04 03:11:22.897503 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.897336    6019 scope.go:122] "RemoveContainer" containerID="ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"
Nov 04 03:11:22.911802 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.911775    6019 scope.go:122] "RemoveContainer" containerID="ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"
Nov 04 03:11:22.912468 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:22.912446    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd\": container with ID starting with ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd not found: ID does not exist" containerID="ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"
Nov 04 03:11:22.912585 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.912476    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd"} err="failed to get container status \"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd\": rpc error: code = NotFound desc = could not find container \"ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd\": container with ID starting with ee15c27c29033b7dff1e4e27dc6a9ba98b25870ecba27ecf2b8ce0da7788f1dd not found: ID does not exist"
Nov 04 03:11:22.986122 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.973462    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\" (UniqueName: \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\") pod \"efe32154-a1cf-4ba0-a775-a95448276eae\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") "
Nov 04 03:11:22.986122 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.973585    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") pod \"efe32154-a1cf-4ba0-a775-a95448276eae\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") "
Nov 04 03:11:22.986122 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.973666    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\") pod \"efe32154-a1cf-4ba0-a775-a95448276eae\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") "
Nov 04 03:11:22.986122 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.973698    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\") pod \"efe32154-a1cf-4ba0-a775-a95448276eae\" (UID: \"efe32154-a1cf-4ba0-a775-a95448276eae\") "
Nov 04 03:11:22.991909 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:22.991860    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca" pod "efe32154-a1cf-4ba0-a775-a95448276eae" (UID: "efe32154-a1cf-4ba0-a775-a95448276eae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:23.017596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.017320    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls" pod "efe32154-a1cf-4ba0-a775-a95448276eae" (UID: "efe32154-a1cf-4ba0-a775-a95448276eae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:23.032291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.032206    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk" pod "efe32154-a1cf-4ba0-a775-a95448276eae" (UID: "efe32154-a1cf-4ba0-a775-a95448276eae"). InnerVolumeSpecName "kube-api-access-5fqlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:23.051103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.051045    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs" pod "efe32154-a1cf-4ba0-a775-a95448276eae" (UID: "efe32154-a1cf-4ba0-a775-a95448276eae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:23.088903 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.088870    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fqlk\" (UniqueName: \"kubernetes.io/projected/efe32154-a1cf-4ba0-a775-a95448276eae-kube-api-access-5fqlk\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:23.088903 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.088899    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:23.088903 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.088909    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe32154-a1cf-4ba0-a775-a95448276eae-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:23.089131 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.088918    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe32154-a1cf-4ba0-a775-a95448276eae-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:23.314543 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.314008    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 03:11:23.328467 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.328431    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe4e0c-54b8c8b98d-868fj"]
Nov 04 03:11:23.742509 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.742423    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:23.893295 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:23.893260    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:24.757294 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:24.757262    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:24.909418 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:24.909335    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:24.983150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:24.983107    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efe32154-a1cf-4ba0-a775-a95448276eae" path="/var/lib/kubelet/pods/efe32154-a1cf-4ba0-a775-a95448276eae/volumes"
Nov 04 03:11:25.888462 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:25.888428    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:11:26.101842 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.101373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:26.252344 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.252315    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:26.593112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.593004    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" containerID="cri-o://debc8e129cf4ecce991a21425d1942595f0dad2b0c19b7cc95b4230d3e364761" gracePeriod=45
Nov 04 03:11:26.633836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.633590    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" containerID="cri-o://ad5cd9c0429c606e3b5bb227700fd87e3a11c8d38553ff3d754c2a0ac691d8b8" gracePeriod=45
Nov 04 03:11:26.693607 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.693565    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": dial tcp 10.132.0.48:8000: connect: connection refused"
Nov 04 03:11:26.946358 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.946268    6019 generic.go:356] "Generic (PLEG): container finished" podID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerID="ad5cd9c0429c606e3b5bb227700fd87e3a11c8d38553ff3d754c2a0ac691d8b8" exitCode=0
Nov 04 03:11:26.946983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.946956    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" event={"ID":"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5","Type":"ContainerDied","Data":"ad5cd9c0429c606e3b5bb227700fd87e3a11c8d38553ff3d754c2a0ac691d8b8"}
Nov 04 03:11:26.949329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.949304    6019 generic.go:356] "Generic (PLEG): container finished" podID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerID="debc8e129cf4ecce991a21425d1942595f0dad2b0c19b7cc95b4230d3e364761" exitCode=0
Nov 04 03:11:26.949529 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:26.949512    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" event={"ID":"7ff179f5-d5b5-4c2c-875e-28713181605f","Type":"ContainerDied","Data":"debc8e129cf4ecce991a21425d1942595f0dad2b0c19b7cc95b4230d3e364761"}
Nov 04 03:11:27.124086 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.124041    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 03:11:27.186256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.186229    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 03:11:27.328329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.328296    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.328619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.328597    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.328763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.328744    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.328900 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.328883    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.329052 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329037    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.329234 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329217    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\" (UniqueName: \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.329424 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329409    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.329558 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329542    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\" (UniqueName: \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.329694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329678    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.329821 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329805    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\" (UniqueName: \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\") pod \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\" (UID: \"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5\") "
Nov 04 03:11:27.329980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.329963    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.330162 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.330144    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\") pod \"7ff179f5-d5b5-4c2c-875e-28713181605f\" (UID: \"7ff179f5-d5b5-4c2c-875e-28713181605f\") "
Nov 04 03:11:27.331012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.330989    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.334802 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.334614    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:27.335139 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.334945    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.335520 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.335276    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.338006 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.337755    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.340460 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.340286    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.340776 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.340605    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.341186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.340972    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.341355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.341321    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:27.341982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.341947    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:27.343665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.343630    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m" pod "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" (UID: "3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5"). InnerVolumeSpecName "kube-api-access-5gt8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:27.344909 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.344888    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj" pod "7ff179f5-d5b5-4c2c-875e-28713181605f" (UID: "7ff179f5-d5b5-4c2c-875e-28713181605f"). InnerVolumeSpecName "kube-api-access-r7gpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:27.431974 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.431941    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.431974 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.431972    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.431974 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.431983    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.431991    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432002    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432009    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ff179f5-d5b5-4c2c-875e-28713181605f-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432016    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432025    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7gpj\" (UniqueName: \"kubernetes.io/projected/7ff179f5-d5b5-4c2c-875e-28713181605f-kube-api-access-r7gpj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432035    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432045    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gt8m\" (UniqueName: \"kubernetes.io/projected/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5-kube-api-access-5gt8m\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432055    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.432310 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.432064    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ff179f5-d5b5-4c2c-875e-28713181605f-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:27.984355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.984253    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm" event={"ID":"3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5","Type":"ContainerDied","Data":"14b308bad74c0290a2fe329d463d444f7bc902a754028f243b26be795fb76765"}
Nov 04 03:11:27.985035 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.984922    6019 scope.go:122] "RemoveContainer" containerID="ad5cd9c0429c606e3b5bb227700fd87e3a11c8d38553ff3d754c2a0ac691d8b8"
Nov 04 03:11:27.985424 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:27.985324    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"
Nov 04 03:11:28.032468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.027936    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" event={"ID":"7ff179f5-d5b5-4c2c-875e-28713181605f","Type":"ContainerDied","Data":"1fdce140cc3ce40fa0b7e7f7430f47758a1ddfdba6deeafe86243e64846746c9"}
Nov 04 03:11:28.032468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.027994    6019 scope.go:122] "RemoveContainer" containerID="debc8e129cf4ecce991a21425d1942595f0dad2b0c19b7cc95b4230d3e364761"
Nov 04 03:11:28.032468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.028139    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"
Nov 04 03:11:28.039578 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.039033    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:28.083095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.081165    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Nov 04 03:11:28.208275 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.208242    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 03:11:28.212740 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.212715    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/premium-w6-ebb7fd-kserve-75dd4dc966-wghjv"]
Nov 04 03:11:28.325609 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.325571    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 03:11:28.329381 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.329353    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ebb7fd/simulator-w6-ebb7fd-kserve-67b777f9f5-x4qdm"]
Nov 04 03:11:28.983698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.983658    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" path="/var/lib/kubelet/pods/3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5/volumes"
Nov 04 03:11:28.984147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:28.984132    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" path="/var/lib/kubelet/pods/7ff179f5-d5b5-4c2c-875e-28713181605f/volumes"
Nov 04 03:11:29.296385 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.296348    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:29.466267 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.466222    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 03:11:29.628169 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.628086    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" containerID="cri-o://861e78ddd4c8e7daafdcae1cd7f234fda8f86712525804c97c24548bec139232" gracePeriod=45
Nov 04 03:11:29.681129 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.676259    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" containerID="cri-o://958b0a3dc2d5b810262c8c098469568a392e2ead565fe650e147c4a0a6c69879" gracePeriod=45
Nov 04 03:11:29.690252 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.690224    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:29.904929 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:29.904493    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.052290    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.052645    6019 generic.go:356] "Generic (PLEG): container finished" podID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerID="861e78ddd4c8e7daafdcae1cd7f234fda8f86712525804c97c24548bec139232" exitCode=0
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.052711    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" event={"ID":"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2","Type":"ContainerDied","Data":"861e78ddd4c8e7daafdcae1cd7f234fda8f86712525804c97c24548bec139232"}
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.052752    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" event={"ID":"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2","Type":"ContainerDied","Data":"0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed"}
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.052764    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed"
Nov 04 03:11:30.054119 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.053334    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 03:11:30.054433 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.054415    6019 generic.go:356] "Generic (PLEG): container finished" podID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerID="958b0a3dc2d5b810262c8c098469568a392e2ead565fe650e147c4a0a6c69879" exitCode=0
Nov 04 03:11:30.054500 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.054489    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" event={"ID":"140ff011-9fdc-4687-95e4-d56d141a8ac2","Type":"ContainerDied","Data":"958b0a3dc2d5b810262c8c098469568a392e2ead565fe650e147c4a0a6c69879"}
Nov 04 03:11:30.060140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.060122    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 03:11:30.073594 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073573    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.073719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073614    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.073719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073680    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.073719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073707    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.073919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073743    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.073919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073769    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\" (UniqueName: \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.073919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073820    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.073919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073844    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.073919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073877    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073948    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073960    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.073981    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074054    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074148    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\" (UniqueName: \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\") pod \"140ff011-9fdc-4687-95e4-d56d141a8ac2\" (UID: \"140ff011-9fdc-4687-95e4-d56d141a8ac2\") "
Nov 04 03:11:30.074205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074188    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\" (UniqueName: \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.074496 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074292    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\") pod \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\" (UID: \"2be3a14c-bdaf-4cd7-aa96-68184d60d8f2\") "
Nov 04 03:11:30.074496 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074382    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074806    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.074973    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.075174    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.075193    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.075207    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.075274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.075220    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.076759 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.076723    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:30.076882 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.076827    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.076882 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.076841    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:11:30.076998 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.076959    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:30.077188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.077164    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq" pod "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" (UID: "2be3a14c-bdaf-4cd7-aa96-68184d60d8f2"). InnerVolumeSpecName "kube-api-access-8p5gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:30.078959 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.078938    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp" pod "140ff011-9fdc-4687-95e4-d56d141a8ac2" (UID: "140ff011-9fdc-4687-95e4-d56d141a8ac2"). InnerVolumeSpecName "kube-api-access-cx8fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:30.176177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176097    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176135    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176149    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/140ff011-9fdc-4687-95e4-d56d141a8ac2-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176162    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/140ff011-9fdc-4687-95e4-d56d141a8ac2-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176176    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176456 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176191    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx8fp\" (UniqueName: \"kubernetes.io/projected/140ff011-9fdc-4687-95e4-d56d141a8ac2-kube-api-access-cx8fp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.176456 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.176208    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p5gq\" (UniqueName: \"kubernetes.io/projected/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2-kube-api-access-8p5gq\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:30.232038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.231999    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:30.943253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.943161    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 04 03:11:30.982136 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:30.980162    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 04 03:11:31.039783 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:31.039749    6019 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2be3a14c_bdaf_4cd7_aa96_68184d60d8f2.slice/crio-0ef8984224bec8ee8cce377d5cd48e8d26a03c810a56d83fd43cdc680fa262ed\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod140ff011_9fdc_4687_95e4_d56d141a8ac2.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod140ff011_9fdc_4687_95e4_d56d141a8ac2.slice/crio-e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583\": RecentStats: unable to find data in memory cache]"
Nov 04 03:11:31.061037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.061010    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"
Nov 04 03:11:31.061188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.061043    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp" event={"ID":"140ff011-9fdc-4687-95e4-d56d141a8ac2","Type":"ContainerDied","Data":"e5de70a7f1a85cf9c85bb0160e86f63dabda6976be47e2c3c006a98bd510c583"}
Nov 04 03:11:31.061188 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.061123    6019 scope.go:122] "RemoveContainer" containerID="958b0a3dc2d5b810262c8c098469568a392e2ead565fe650e147c4a0a6c69879"
Nov 04 03:11:31.061345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.061330    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"
Nov 04 03:11:31.232154 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.229757    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 03:11:31.254968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.254921    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:31.278171 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.278141    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/premium-w3-cdcf67-kserve-5c579cbdd8-dst7n"]
Nov 04 03:11:31.352048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.352013    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 03:11:31.357699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.357669    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-cdcf67/simulator-w3-cdcf67-kserve-846669b9cc-9lcdp"]
Nov 04 03:11:31.732844 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.732807    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 03:11:31.733159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.733058    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy" containerID="cri-o://3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d" gracePeriod=30
Nov 04 03:11:31.769612 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.769571    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:31.925427 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:31.925398    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:32.725441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:32.725383    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:32.981973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:32.981895    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" path="/var/lib/kubelet/pods/140ff011-9fdc-4687-95e4-d56d141a8ac2/volumes"
Nov 04 03:11:32.982380 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:32.982366    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" path="/var/lib/kubelet/pods/2be3a14c-bdaf-4cd7-aa96-68184d60d8f2/volumes"
Nov 04 03:11:33.089137 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:33.089100    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:33.244782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:33.244687    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:33.248547 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:33.248516    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 03:11:33.248781 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:33.248743    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy" containerID="cri-o://f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421" gracePeriod=30
Nov 04 03:11:34.467610 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.467570    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" podUID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" containerName="payload-pre-processing" containerID="cri-o://aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4" gracePeriod=25
Nov 04 03:11:34.692316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.692284    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:34.798539 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.798508    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 03:11:34.840740 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.840686    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:34.875161 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.875131    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\" (UniqueName: \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\") pod \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") "
Nov 04 03:11:34.875334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.875234    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\") pod \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\" (UID: \"588ab8e8-fc03-4c32-bd6c-af484aa51afc\") "
Nov 04 03:11:34.875588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.875567    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume" pod "588ab8e8-fc03-4c32-bd6c-af484aa51afc" (UID: "588ab8e8-fc03-4c32-bd6c-af484aa51afc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:34.877494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.877473    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l" pod "588ab8e8-fc03-4c32-bd6c-af484aa51afc" (UID: "588ab8e8-fc03-4c32-bd6c-af484aa51afc"). InnerVolumeSpecName "kube-api-access-2l82l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:34.976975 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.976878    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588ab8e8-fc03-4c32-bd6c-af484aa51afc-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:34.976975 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.976909    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l82l\" (UniqueName: \"kubernetes.io/projected/588ab8e8-fc03-4c32-bd6c-af484aa51afc-kube-api-access-2l82l\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:34.984581 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:34.984554    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:35.084486 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.084449    6019 generic.go:356] "Generic (PLEG): container finished" podID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" containerID="aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4" exitCode=0
Nov 04 03:11:35.084680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.084501    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"
Nov 04 03:11:35.084680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.084533    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" event={"ID":"588ab8e8-fc03-4c32-bd6c-af484aa51afc","Type":"ContainerDied","Data":"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"}
Nov 04 03:11:35.084680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.084597    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n" event={"ID":"588ab8e8-fc03-4c32-bd6c-af484aa51afc","Type":"ContainerDied","Data":"e053daffb0a8991c02f72699222bf6c21ecd4abcb787c452a93e6dc2679771db"}
Nov 04 03:11:35.084680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.084624    6019 scope.go:122] "RemoveContainer" containerID="aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"
Nov 04 03:11:35.094040 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.094019    6019 scope.go:122] "RemoveContainer" containerID="aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"
Nov 04 03:11:35.094400 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:35.094371    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4\": container with ID starting with aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4 not found: ID does not exist" containerID="aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"
Nov 04 03:11:35.094501 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.094411    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4"} err="failed to get container status \"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4\": rpc error: code = NotFound desc = could not find container \"aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4\": container with ID starting with aa378cbde150dda105e46e386ebfeb48954966f6a9bee2302f6d2e292a32c8e4 not found: ID does not exist"
Nov 04 03:11:35.245149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.240023    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:35.261116 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.261043    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 03:11:35.261633 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.261606    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n"]
Nov 04 03:11:35.908508 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:35.908476    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:36.603332 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:36.603286    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:11:36.853924 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:36.853446    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:36.982871 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:36.982839    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" path="/var/lib/kubelet/pods/588ab8e8-fc03-4c32-bd6c-af484aa51afc/volumes"
Nov 04 03:11:37.021210 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.021188    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 03:11:37.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.107480    6019 generic.go:356] "Generic (PLEG): container finished" podID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerID="3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d" exitCode=0
Nov 04 03:11:37.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.107528    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"
Nov 04 03:11:37.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.107528    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" event={"ID":"eb8ab58a-05e8-4c31-97b8-726adf35bd57","Type":"ContainerDied","Data":"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"}
Nov 04 03:11:37.107755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.107581    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p" event={"ID":"eb8ab58a-05e8-4c31-97b8-726adf35bd57","Type":"ContainerDied","Data":"2d7da5bc6cde6cbc005a4a6016ad4def83ab272c3fbcd7b3152f719603f4c088"}
Nov 04 03:11:37.107755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.107612    6019 scope.go:122] "RemoveContainer" containerID="3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"
Nov 04 03:11:37.109885 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.109861    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.109988 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.109902    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.109988 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.109967    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110017    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110053    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110112 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110102    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110158    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110253 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110191    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110358 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110262    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:37.110358 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110298    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\") pod \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\" (UID: \"eb8ab58a-05e8-4c31-97b8-726adf35bd57\") "
Nov 04 03:11:37.110632 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110579    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:37.110751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110724    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:37.110833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110773    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:37.110833 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110807    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:37.110976 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110959    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.111038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110982    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.111038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.110997    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.111038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.111011    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.111038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.111025    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.112307 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.112274    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:37.112690 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.112664    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:11:37.113195 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.113173    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:37.113305 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.113216    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8" pod "eb8ab58a-05e8-4c31-97b8-726adf35bd57" (UID: "eb8ab58a-05e8-4c31-97b8-726adf35bd57"). InnerVolumeSpecName "kube-api-access-rb6d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:37.135474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.135451    6019 scope.go:122] "RemoveContainer" containerID="3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"
Nov 04 03:11:37.135827 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:37.135809    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d\": container with ID starting with 3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d not found: ID does not exist" containerID="3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"
Nov 04 03:11:37.135894 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.135836    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d"} err="failed to get container status \"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d\": rpc error: code = NotFound desc = could not find container \"3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d\": container with ID starting with 3318d0916647929bb683b5b0c7dcefdb13a8b45ea31715a2c507f4998357be3d not found: ID does not exist"
Nov 04 03:11:37.212018 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.211989    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.212018 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.212019    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.212239 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.212033    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8ab58a-05e8-4c31-97b8-726adf35bd57-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.212239 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.212049    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb6d8\" (UniqueName: \"kubernetes.io/projected/eb8ab58a-05e8-4c31-97b8-726adf35bd57-kube-api-access-rb6d8\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:37.527396 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.527363    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 03:11:37.532766 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.532739    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p"]
Nov 04 03:11:37.601355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:37.601321    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:38.499148 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.499123    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 03:11:38.568506 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.568440    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:38.628565 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628539    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628589    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628627    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628663    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628741 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628719    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628762    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628831    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628885    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.628955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628904    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:38.628955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.628934    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\") pod \"b2042e3c-08cb-4694-92ed-e22b87d310cc\" (UID: \"b2042e3c-08cb-4694-92ed-e22b87d310cc\") "
Nov 04 03:11:38.629233 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629045    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:38.629233 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629099    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:38.629233 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629181    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:38.629449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629425    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.629574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629451    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.629574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629466    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.629574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629478    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.629750 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.629660    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:38.631103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.631063    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:38.631187 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.631114    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:11:38.631693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.631674    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:38.631693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.631683    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz" pod "b2042e3c-08cb-4694-92ed-e22b87d310cc" (UID: "b2042e3c-08cb-4694-92ed-e22b87d310cc"). InnerVolumeSpecName "kube-api-access-ntvhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:38.730103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.730048    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.730103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.730101    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.730103 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.730111    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2042e3c-08cb-4694-92ed-e22b87d310cc-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.730318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.730121    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntvhz\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-kube-api-access-ntvhz\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.730318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.730130    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2042e3c-08cb-4694-92ed-e22b87d310cc-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:38.981485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:38.981394    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" path="/var/lib/kubelet/pods/eb8ab58a-05e8-4c31-97b8-726adf35bd57/volumes"
Nov 04 03:11:39.121203 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.121168    6019 generic.go:356] "Generic (PLEG): container finished" podID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerID="f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421" exitCode=0
Nov 04 03:11:39.121394 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.121227    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"
Nov 04 03:11:39.121394 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.121259    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" event={"ID":"b2042e3c-08cb-4694-92ed-e22b87d310cc","Type":"ContainerDied","Data":"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"}
Nov 04 03:11:39.121394 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.121343    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6" event={"ID":"b2042e3c-08cb-4694-92ed-e22b87d310cc","Type":"ContainerDied","Data":"bc87fe1d391c430de90e94aad8ccd9fb1b7b5a1f8e60a63a54ea63276355fa56"}
Nov 04 03:11:39.121394 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.121371    6019 scope.go:122] "RemoveContainer" containerID="f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"
Nov 04 03:11:39.133522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.133478    6019 scope.go:122] "RemoveContainer" containerID="f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"
Nov 04 03:11:39.134103 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:39.134054    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421\": container with ID starting with f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421 not found: ID does not exist" containerID="f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"
Nov 04 03:11:39.134197 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.134114    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421"} err="failed to get container status \"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421\": rpc error: code = NotFound desc = could not find container \"f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421\": container with ID starting with f2f09c90b2d2f972ed83b11051f3e2a308c9d6cca68ba45422672b4ca3eeb421 not found: ID does not exist"
Nov 04 03:11:39.284408 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.284376    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 03:11:39.288371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:39.288347    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6"]
Nov 04 03:11:40.691826 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:40.691788    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:40.982523 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:40.982448    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" path="/var/lib/kubelet/pods/b2042e3c-08cb-4694-92ed-e22b87d310cc/volumes"
Nov 04 03:11:41.966404 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:41.966373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:42.110287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:42.110255    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:42.254956 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:42.254916    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:43.608267 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608237    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:11:43.608722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608683    6019 cpu_manager.go:414] "containerMap: removing container" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main"
Nov 04 03:11:43.608722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608695    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main"
Nov 04 03:11:43.608722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608716    6019 cpu_manager.go:414] "containerMap: removing container" podUID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" containerName="payload-pre-processing"
Nov 04 03:11:43.608722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608722    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" containerName="payload-pre-processing"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608729    6019 cpu_manager.go:414] "containerMap: removing container" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608734    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608747    6019 cpu_manager.go:414] "containerMap: removing container" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608751    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608764    6019 cpu_manager.go:414] "containerMap: removing container" podUID="efe32154-a1cf-4ba0-a775-a95448276eae" containerName="maas-api"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608769    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efe32154-a1cf-4ba0-a775-a95448276eae" containerName="maas-api"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608778    6019 cpu_manager.go:414] "containerMap: removing container" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608782    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608798    6019 cpu_manager.go:414] "containerMap: removing container" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608803    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608811    6019 cpu_manager.go:414] "containerMap: removing container" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main"
Nov 04 03:11:43.608858 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608815    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608872    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b5bc6a6-9bb7-4535-9952-e6ffdac4f6d5" containerName="main"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608881    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="140ff011-9fdc-4687-95e4-d56d141a8ac2" containerName="main"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608888    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ff179f5-d5b5-4c2c-875e-28713181605f" containerName="main"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608897    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="588ab8e8-fc03-4c32-bd6c-af484aa51afc" containerName="payload-pre-processing"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608903    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2042e3c-08cb-4694-92ed-e22b87d310cc" containerName="istio-proxy"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608910    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb8ab58a-05e8-4c31-97b8-726adf35bd57" containerName="istio-proxy"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608917    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="efe32154-a1cf-4ba0-a775-a95448276eae" containerName="maas-api"
Nov 04 03:11:43.609274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.608923    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="2be3a14c-bdaf-4cd7-aa96-68184d60d8f2" containerName="main"
Nov 04 03:11:43.613041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.613024    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.678773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.678740    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt8zw\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.678773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.678774    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.678966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.678890    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.678966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.678937    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.679047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.679021    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.679135 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.679110    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.679199 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.679179    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.679262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.679242    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.679320 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.679267    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.716696 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.716669    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:11:43.780437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780404    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780445    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780502    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt8zw\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780528    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780734 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780608    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780734 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780662    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780734 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780689    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780733    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780807    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.780889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780874    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.781047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.780975    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.781200 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.781180    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.781271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.781211    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.781335 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.781318    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.783129 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.783106    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.783359 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.783340    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.841194 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.841162    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.841994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.841977    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt8zw\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\") pod \"e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") " pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:43.927690 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:43.927618    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:44.355220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:44.355193    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:11:44.357837 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:11:44.357813    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16bd1a17_2936_4c48_a301_d256389b3753.slice/crio-8d1c80e76ac786becc86199acc37dc3e184294b938ad1dfc91d1613e1ae573da WatchSource:0}: Error finding container 8d1c80e76ac786becc86199acc37dc3e184294b938ad1dfc91d1613e1ae573da: Status 404 returned error can't find the container with id 8d1c80e76ac786becc86199acc37dc3e184294b938ad1dfc91d1613e1ae573da
Nov 04 03:11:44.360181 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:44.360140    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:11:44.360300 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:44.360228    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:11:44.360300 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:44.360254    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:11:45.178476 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.178440    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" event={"ID":"16bd1a17-2936-4c48-a301-d256389b3753","Type":"ContainerStarted","Data":"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"}
Nov 04 03:11:45.178877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.178500    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" event={"ID":"16bd1a17-2936-4c48-a301-d256389b3753","Type":"ContainerStarted","Data":"8d1c80e76ac786becc86199acc37dc3e184294b938ad1dfc91d1613e1ae573da"}
Nov 04 03:11:45.397676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.397617    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podStartSLOduration=2.397601359 podStartE2EDuration="2.397601359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:11:45.346925168 +0000 UTC m=+2260.487938599" watchObservedRunningTime="2026-11-04 03:11:45.397601359 +0000 UTC m=+2260.538614827"
Nov 04 03:11:45.927962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.927933    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:45.929449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.929428    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:45.929504 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:45.929474    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:46.928060 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:46.928026    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:46.928479 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:46.928115    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:47.212632 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:47.212552    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:47.928204 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:47.928170    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:47.928680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:47.928228    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:48.934096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:48.930499    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:48.934096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:48.930581    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:49.740628 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:49.735509    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:49.877249 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:49.877217    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:49.928057 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:49.928027    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:49.928227 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:49.928088    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:50.928756 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:50.928716    6019 patch_prober.go:29] interesting pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 04 03:11:50.929205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:50.928789    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 04 03:11:51.934634 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:51.934602    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:51.935158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:51.935126    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:51.936124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:51.936094    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:51.978639 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:51.978612    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:52.124390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.124355    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:52.350305 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.350267    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:52.514782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.514716    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:52.831513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.831477    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:11:52.855752 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.855718    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:52.868143 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.868111    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 03:11:52.868387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:52.868359    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy" containerID="cri-o://fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83" gracePeriod=30
Nov 04 03:11:53.012313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:53.012142    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:53.226257 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:53.226176    6019 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" secret="" err="secret \"e2e-worker-w6-ebb7fd-openshift-default-dockercfg-k9smx\" not found"
Nov 04 03:11:54.231468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:54.231417    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy" containerID="cri-o://361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c" gracePeriod=30
Nov 04 03:11:55.816906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:55.816873    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:57.628199 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:57.628170    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:57.794488 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:57.794450    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:57.940283 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:57.940254    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:58.136160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.136135    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 03:11:58.207942 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.207855    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.207942 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.207928    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.207977    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208019    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208059    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208104    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208164    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208204    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:58.208564 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208260    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208564 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208290    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\") pod \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\" (UID: \"fdde55a0-318f-45be-ab19-41f2ea1e8f94\") "
Nov 04 03:11:58.208564 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208438    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:58.208564 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208502    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:58.208749 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208654    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:58.208800 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208754    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:58.208878 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208863    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.208917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208884    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.208917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208898    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.208917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208912    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.209039 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.208925    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.210623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.210596    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:58.210623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.210601    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "kube-api-access-n49rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:58.210815 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.210778    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:58.210981 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.210959    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo" pod "fdde55a0-318f-45be-ab19-41f2ea1e8f94" (UID: "fdde55a0-318f-45be-ab19-41f2ea1e8f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:11:58.256313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.256279    6019 generic.go:356] "Generic (PLEG): container finished" podID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerID="fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83" exitCode=0
Nov 04 03:11:58.256482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.256347    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"
Nov 04 03:11:58.256482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.256371    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" event={"ID":"fdde55a0-318f-45be-ab19-41f2ea1e8f94","Type":"ContainerDied","Data":"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"}
Nov 04 03:11:58.256482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.256435    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql" event={"ID":"fdde55a0-318f-45be-ab19-41f2ea1e8f94","Type":"ContainerDied","Data":"8628b1714ff73a9880c8977748e05612627ab09f3d850189a0506762a5807b82"}
Nov 04 03:11:58.256482 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.256464    6019 scope.go:122] "RemoveContainer" containerID="fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"
Nov 04 03:11:58.268164 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.268138    6019 scope.go:122] "RemoveContainer" containerID="fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"
Nov 04 03:11:58.268496 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:11:58.268467    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83\": container with ID starting with fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83 not found: ID does not exist" containerID="fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"
Nov 04 03:11:58.268591 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.268503    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83"} err="failed to get container status \"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83\": rpc error: code = NotFound desc = could not find container \"fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83\": container with ID starting with fc6ede5727ca05859e0baae01c74743a7b65bec9420698a32de0e94875977a83 not found: ID does not exist"
Nov 04 03:11:58.309757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.309725    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.309757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.309759    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.310028 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.309776    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n49rp\" (UniqueName: \"kubernetes.io/projected/fdde55a0-318f-45be-ab19-41f2ea1e8f94-kube-api-access-n49rp\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.310028 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.309791    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdde55a0-318f-45be-ab19-41f2ea1e8f94-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:58.420257 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.420225    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 03:11:58.424116 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.424089    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql"]
Nov 04 03:11:58.981257 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:58.981223    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" path="/var/lib/kubelet/pods/fdde55a0-318f-45be-ab19-41f2ea1e8f94/volumes"
Nov 04 03:11:59.092313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.092278    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:11:59.491387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.491362    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:11:59.521519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521487    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521621    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521669    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521707    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521741    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521798    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.521836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521831    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.522286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521933    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.522286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.521971    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\") pod \"16bd1a17-2936-4c48-a301-d256389b3753\" (UID: \"16bd1a17-2936-4c48-a301-d256389b3753\") "
Nov 04 03:11:59.522286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522226    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:11:59.522721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522505    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:59.522721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522609    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16bd1a17-2936-4c48-a301-d256389b3753-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.522721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522641    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:59.522721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522716    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:59.523056 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.522747    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:59.524091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.524034    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:59.524242 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.524219    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:11:59.524825 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.524805    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:11:59.525028 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.525012    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw" pod "16bd1a17-2936-4c48-a301-d256389b3753" (UID: "16bd1a17-2936-4c48-a301-d256389b3753"). InnerVolumeSpecName "kube-api-access-nt8zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:11:59.623422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623390    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt8zw\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-kube-api-access-nt8zw\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623419    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623430    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16bd1a17-2936-4c48-a301-d256389b3753-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623438    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16bd1a17-2936-4c48-a301-d256389b3753-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623448    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623456    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623466    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:11:59.623666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:11:59.623477    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16bd1a17-2936-4c48-a301-d256389b3753-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:12:00.269589 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.269551    6019 generic.go:356] "Generic (PLEG): container finished" podID="16bd1a17-2936-4c48-a301-d256389b3753" containerID="361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c" exitCode=0
Nov 04 03:12:00.270134 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.269608    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"
Nov 04 03:12:00.270134 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.269645    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" event={"ID":"16bd1a17-2936-4c48-a301-d256389b3753","Type":"ContainerDied","Data":"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"}
Nov 04 03:12:00.270134 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.269725    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj" event={"ID":"16bd1a17-2936-4c48-a301-d256389b3753","Type":"ContainerDied","Data":"8d1c80e76ac786becc86199acc37dc3e184294b938ad1dfc91d1613e1ae573da"}
Nov 04 03:12:00.270134 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.269760    6019 scope.go:122] "RemoveContainer" containerID="361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"
Nov 04 03:12:00.280643 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.280626    6019 scope.go:122] "RemoveContainer" containerID="361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"
Nov 04 03:12:00.280906 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:12:00.280884    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c\": container with ID starting with 361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c not found: ID does not exist" containerID="361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"
Nov 04 03:12:00.281000 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.280912    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c"} err="failed to get container status \"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c\": rpc error: code = NotFound desc = could not find container \"361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c\": container with ID starting with 361c3d596b9b968bff579f1a958fccd21acb765bb3bac36f47876026c480945c not found: ID does not exist"
Nov 04 03:12:00.433579 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.433548    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:12:00.483244 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.483212    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj"]
Nov 04 03:12:00.981272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:00.981237    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16bd1a17-2936-4c48-a301-d256389b3753" path="/var/lib/kubelet/pods/16bd1a17-2936-4c48-a301-d256389b3753/volumes"
Nov 04 03:12:02.095094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:02.095026    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:04.001147 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:04.001113    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:05.658049 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:05.658014    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:05.804009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:05.803978    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:06.484784 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:06.484751    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:06.610144 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:12:06.610105    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:12:07.270895 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:07.270865    6019 scope.go:122] "RemoveContainer" containerID="861e78ddd4c8e7daafdcae1cd7f234fda8f86712525804c97c24548bec139232"
Nov 04 03:12:07.280120 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:07.280100    6019 scope.go:122] "RemoveContainer" containerID="7c504bbd2c5b36de00d487ece680e8627ffeb02bb790baecd05834d2cc7fec0b"
Nov 04 03:12:09.015514 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:09.015479    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:10.797229 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:10.797185    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:10.941324 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:10.941286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:11.569199 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:11.569165    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:12.699215 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:12.699178    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:12.853410 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:12.853373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:13.381756 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:13.381725    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:13.530804 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:13.530764    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:13.677445 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:13.677373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:14.432701 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:14.432668    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:15.079353 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:15.079318    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:15.510145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:15.510102    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:19.786699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:19.786659    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:24.690585 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:24.690547    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:29.593400 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:29.593357    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:34.490867 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:34.490830    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:36.603205 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:12:36.603160    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:12:38.925653 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:38.925623    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:39.394878 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:39.394846    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:41.072294 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:41.072260    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:42.926007 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.925978    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:12:42.926670 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926651    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy"
Nov 04 03:12:42.926748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926672    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy"
Nov 04 03:12:42.926748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926695    6019 cpu_manager.go:414] "containerMap: removing container" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy"
Nov 04 03:12:42.926748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926701    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy"
Nov 04 03:12:42.926854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926801    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdde55a0-318f-45be-ab19-41f2ea1e8f94" containerName="istio-proxy"
Nov 04 03:12:42.926854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.926815    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="16bd1a17-2936-4c48-a301-d256389b3753" containerName="istio-proxy"
Nov 04 03:12:42.930170 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.930154    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997526 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997489    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997541    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxzk4\" (UniqueName: \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997655    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997817 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997699    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997855 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997811    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:42.997899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:42.997881    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.032698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.032653    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:43.035364 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.035341    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4cbcf738\"/\"default-dockercfg-hlxl4\""
Nov 04 03:12:43.035364 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.035345    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4cbcf738\"/\"openshift-service-ca.crt\""
Nov 04 03:12:43.035661 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.035646    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4cbcf738\"/\"kube-root-ca.crt\""
Nov 04 03:12:43.035731 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.035686    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4cbcf738\"/\"e2e-lifecycle-model-4cbcf738-kserve-self-signed-certs\""
Nov 04 03:12:43.037292 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.037273    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:12:43.103098 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103042    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103302 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103195    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxzk4\" (UniqueName: \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103302 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103261    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103429 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103302    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103429 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103408    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103485    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.103950 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.103925    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.104915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.104618    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.107343 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.107322    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.112747 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.112725    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.117668 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.117614    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.165402 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.165363    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxzk4\" (UniqueName: \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\") pod \"e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") " pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.242255 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.242178    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:43.621665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:43.620284    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:12:44.409837 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:44.409789    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:44.498672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:44.498642    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" event={"ID":"090014a4-3291-44d8-9784-e151bfd3cb0d","Type":"ContainerStarted","Data":"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"}
Nov 04 03:12:44.498672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:44.498680    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" event={"ID":"090014a4-3291-44d8-9784-e151bfd3cb0d","Type":"ContainerStarted","Data":"ed8b7e9c4c2d5c8666161c9323853d2233a2953a253d2beffdcc2b53b922c445"}
Nov 04 03:12:44.616420 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:44.616361    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" podStartSLOduration=2.616347127 podStartE2EDuration="2.616347127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:12:44.614520852 +0000 UTC m=+2319.755534252" watchObservedRunningTime="2026-11-04 03:12:44.616347127 +0000 UTC m=+2319.757360549"
Nov 04 03:12:45.243329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:45.243299    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:49.585629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:49.585596    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:49.730636 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:49.730600    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:53.242680 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:53.242649    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:53.256341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:53.256316    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:53.559449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:53.559420    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:12:54.073006 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:54.072973    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:54.513245 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:54.513209    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:12:59.403506 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:12:59.403440    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:04.284108 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:04.284005    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:06.603710 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:13:06.603669    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:13:09.183035 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:09.182991    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:14.117503 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:14.117470    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:18.998794 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:18.998763    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:23.892393 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:23.892356    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:28.886522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:28.886487    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:31.231916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.231879    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:13:31.235567 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.235551    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.275973    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.276045    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276367 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.276339    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.276453    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g4fg\" (UniqueName: \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.276487    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.276560 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.276522    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.320627 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.320589    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:31.337995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.337961    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-095081\"/\"test-model-095081-kserve-self-signed-certs\""
Nov 04 03:13:31.337995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.337980    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-095081\"/\"default-dockercfg-jkbn9\""
Nov 04 03:13:31.337995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.337967    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-095081\"/\"openshift-service-ca.crt\""
Nov 04 03:13:31.338269 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.337964    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-095081\"/\"kube-root-ca.crt\""
Nov 04 03:13:31.339902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.339883    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.379952    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380058    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4fg\" (UniqueName: \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380114    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380160    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380239    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380300    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.380818    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.381193 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.381103    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.386895 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.384873    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.396308 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.396241    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.397059 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.397003    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.442752 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.442717    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g4fg\" (UniqueName: \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\") pod \"test-model-095081-kserve-64c84c89d9-qznj8\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") " pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.548144 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.548105    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:31.933371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:31.933280    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:13:31.936467 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:13:31.936439    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbea7cb70_ac11_4f0c_8139_cd4fa3f2bd89.slice/crio-1204bda02b6e461d1dc7c80028092400b2ec8b4c6d5b302cb1e7b7a6af2e75fb WatchSource:0}: Error finding container 1204bda02b6e461d1dc7c80028092400b2ec8b4c6d5b302cb1e7b7a6af2e75fb: Status 404 returned error can't find the container with id 1204bda02b6e461d1dc7c80028092400b2ec8b4c6d5b302cb1e7b7a6af2e75fb
Nov 04 03:13:32.754115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:32.754049    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" event={"ID":"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89","Type":"ContainerStarted","Data":"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"}
Nov 04 03:13:32.754531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:32.754126    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" event={"ID":"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89","Type":"ContainerStarted","Data":"1204bda02b6e461d1dc7c80028092400b2ec8b4c6d5b302cb1e7b7a6af2e75fb"}
Nov 04 03:13:32.871687 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:32.871631    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" podStartSLOduration=1.87161674 podStartE2EDuration="1.87161674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:13:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:13:32.870539442 +0000 UTC m=+2368.011552862" watchObservedRunningTime="2026-11-04 03:13:32.87161674 +0000 UTC m=+2368.012630148"
Nov 04 03:13:33.549033 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:33.548992    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:33.783211 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:33.783178    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:36.605255 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:13:36.605211    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:13:38.684274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:38.684230    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:39.275698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:39.275663    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:40.700798 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.700764    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7027-5a1c78"]
Nov 04 03:13:40.704631 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.704608    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:40.760619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.760585    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7027-5a1c78"]
Nov 04 03:13:40.795953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.795916    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch9wh\" (UniqueName: \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\") pod \"test-curl-7027-5a1c78\" (UID: \"081c7f5a-b033-4c7f-87a8-9254a5b75717\") " pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:40.807260 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.807235    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hzl5n\""
Nov 04 03:13:40.897491 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.897450    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch9wh\" (UniqueName: \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\") pod \"test-curl-7027-5a1c78\" (UID: \"081c7f5a-b033-4c7f-87a8-9254a5b75717\") " pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:40.959000 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:40.958923    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch9wh\" (UniqueName: \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\") pod \"test-curl-7027-5a1c78\" (UID: \"081c7f5a-b033-4c7f-87a8-9254a5b75717\") " pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:41.018511 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.018477    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:41.390168 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.390146    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7027-5a1c78"]
Nov 04 03:13:41.393100 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:13:41.393040    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod081c7f5a_b033_4c7f_87a8_9254a5b75717.slice/crio-f3a3c8ed23881150cefd1c6dc7f470f747aadc14207d73f96e9f4e90be29c8ff WatchSource:0}: Error finding container f3a3c8ed23881150cefd1c6dc7f470f747aadc14207d73f96e9f4e90be29c8ff: Status 404 returned error can't find the container with id f3a3c8ed23881150cefd1c6dc7f470f747aadc14207d73f96e9f4e90be29c8ff
Nov 04 03:13:41.395035 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.395020    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:13:41.428118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.428093    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:41.548321 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.548290    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:41.560842 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.560816    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:41.802469 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.802430    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7027-5a1c78" event={"ID":"081c7f5a-b033-4c7f-87a8-9254a5b75717","Type":"ContainerStarted","Data":"f3a3c8ed23881150cefd1c6dc7f470f747aadc14207d73f96e9f4e90be29c8ff"}
Nov 04 03:13:41.813577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:41.813548    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:13:43.589433 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:43.589375    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:44.820345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:44.820313    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7027-5a1c78" event={"ID":"081c7f5a-b033-4c7f-87a8-9254a5b75717","Type":"ContainerStarted","Data":"6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e"}
Nov 04 03:13:44.933117 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:44.933056    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7027-5a1c78" podStartSLOduration=2.489547371 podStartE2EDuration="4.933042742s" totalImagesPullingTime="2.443495371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:13:44.932479231 +0000 UTC m=+2380.073492638" watchObservedRunningTime="2026-11-04 03:13:44.933042742 +0000 UTC m=+2380.074056148"
Nov 04 03:13:46.474465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:46.474430    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7027-5a1c78"]
Nov 04 03:13:46.478735 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:46.478709    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7027-5a1c78"]
Nov 04 03:13:46.830354 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:46.830316    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7027-5a1c78" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" containerName="test-curl-7027-5a1c78" containerID="cri-o://6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e" gracePeriod=2
Nov 04 03:13:46.881496 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:13:46.881459    6019 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7027-5a1c78\" is forbidden: User \"system:node:ip-10-0-128-201.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-201.us-west-1.compute.internal' and this object" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:48.828295 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:48.828257    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:49.079101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.079024    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:49.191779 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.191753    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\" (UniqueName: \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\") pod \"081c7f5a-b033-4c7f-87a8-9254a5b75717\" (UID: \"081c7f5a-b033-4c7f-87a8-9254a5b75717\") "
Nov 04 03:13:49.194051 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.194026    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh" pod "081c7f5a-b033-4c7f-87a8-9254a5b75717" (UID: "081c7f5a-b033-4c7f-87a8-9254a5b75717"). InnerVolumeSpecName "kube-api-access-ch9wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:13:49.293451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.293425    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch9wh\" (UniqueName: \"kubernetes.io/projected/081c7f5a-b033-4c7f-87a8-9254a5b75717-kube-api-access-ch9wh\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:13:49.846886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.846847    6019 generic.go:356] "Generic (PLEG): container finished" podID="081c7f5a-b033-4c7f-87a8-9254a5b75717" containerID="6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e" exitCode=137
Nov 04 03:13:49.847368 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.846894    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:49.847368 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.846920    6019 scope.go:122] "RemoveContainer" containerID="6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e"
Nov 04 03:13:49.855955 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.855933    6019 scope.go:122] "RemoveContainer" containerID="6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e"
Nov 04 03:13:49.856259 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:13:49.856229    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e\": container with ID starting with 6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e not found: ID does not exist" containerID="6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e"
Nov 04 03:13:49.856326 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:49.856268    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e"} err="failed to get container status \"6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e\": rpc error: code = NotFound desc = could not find container \"6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e\": container with ID starting with 6097721a154b12663e9210a757ec2204c022351148a6bdce3ca2d9100594d71e not found: ID does not exist"
Nov 04 03:13:49.906355 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:13:49.906319    6019 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7027-5a1c78\" is forbidden: User \"system:node:ip-10-0-128-201.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-201.us-west-1.compute.internal' and this object" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" pod="opendatahub/test-curl-7027-5a1c78"
Nov 04 03:13:50.762047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:50.762010    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:50.905018 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:50.904991    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:50.982159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:50.982125    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" path="/var/lib/kubelet/pods/081c7f5a-b033-4c7f-87a8-9254a5b75717/volumes"
Nov 04 03:13:54.087264 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:54.087224    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:58.992903 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:58.992835    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:59.351192 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:59.351151    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:13:59.497732 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:13:59.497704    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:04.017363 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:04.017326    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:05.992901 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:05.992861    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:14:06.005841 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:06.005811    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:14:06.603596 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:14:06.603562    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:14:08.898419 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:08.898383    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:13.787323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:13.787289    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:18.684682 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:18.684635    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:23.599792 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:23.599746    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:28.493738 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:28.493702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:33.401533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:33.401490    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:36.604061 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:14:36.604022    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:14:38.366797 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:38.366759    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:39.633906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:39.633841    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:41.781137 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:41.781098    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:42.450300 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.450265    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:14:42.450550 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.450506    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" podUID="42f270fb-5da0-48c1-9011-64078fb7f5a2" containerName="maas-api" containerID="cri-o://1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee" gracePeriod=30
Nov 04 03:14:42.553958 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.553844    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:42.758783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.758755    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:14:42.775422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.775382    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") pod \"42f270fb-5da0-48c1-9011-64078fb7f5a2\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") "
Nov 04 03:14:42.775575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.775539    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\") pod \"42f270fb-5da0-48c1-9011-64078fb7f5a2\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") "
Nov 04 03:14:42.775643 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.775615    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\") pod \"42f270fb-5da0-48c1-9011-64078fb7f5a2\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") "
Nov 04 03:14:42.775726 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.775711    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\" (UniqueName: \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\") pod \"42f270fb-5da0-48c1-9011-64078fb7f5a2\" (UID: \"42f270fb-5da0-48c1-9011-64078fb7f5a2\") "
Nov 04 03:14:42.776236 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.776205    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca" pod "42f270fb-5da0-48c1-9011-64078fb7f5a2" (UID: "42f270fb-5da0-48c1-9011-64078fb7f5a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:14:42.777912 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.777887    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5" pod "42f270fb-5da0-48c1-9011-64078fb7f5a2" (UID: "42f270fb-5da0-48c1-9011-64078fb7f5a2"). InnerVolumeSpecName "kube-api-access-qmbm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:14:42.778232 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.778214    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs" pod "42f270fb-5da0-48c1-9011-64078fb7f5a2" (UID: "42f270fb-5da0-48c1-9011-64078fb7f5a2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:14:42.778331 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.778245    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls" pod "42f270fb-5da0-48c1-9011-64078fb7f5a2" (UID: "42f270fb-5da0-48c1-9011-64078fb7f5a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:14:42.877407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.877364    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmbm5\" (UniqueName: \"kubernetes.io/projected/42f270fb-5da0-48c1-9011-64078fb7f5a2-kube-api-access-qmbm5\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:14:42.877407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.877403    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:14:42.877944 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.877417    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f270fb-5da0-48c1-9011-64078fb7f5a2-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:14:42.877944 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:42.877458    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f270fb-5da0-48c1-9011-64078fb7f5a2-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:14:43.140612 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.140582    6019 generic.go:356] "Generic (PLEG): container finished" podID="42f270fb-5da0-48c1-9011-64078fb7f5a2" containerID="1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee" exitCode=0
Nov 04 03:14:43.140787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.140631    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"
Nov 04 03:14:43.140787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.140657    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" event={"ID":"42f270fb-5da0-48c1-9011-64078fb7f5a2","Type":"ContainerDied","Data":"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"}
Nov 04 03:14:43.140787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.140697    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf" event={"ID":"42f270fb-5da0-48c1-9011-64078fb7f5a2","Type":"ContainerDied","Data":"d3da91e30a89d03b8fd291b346f0c0a541adccd39aa22a1d2cfebcb9e12205d4"}
Nov 04 03:14:43.140787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.140718    6019 scope.go:122] "RemoveContainer" containerID="1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"
Nov 04 03:14:43.150428 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.150398    6019 scope.go:122] "RemoveContainer" containerID="1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"
Nov 04 03:14:43.150685 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:14:43.150666    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee\": container with ID starting with 1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee not found: ID does not exist" containerID="1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"
Nov 04 03:14:43.150734 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.150694    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee"} err="failed to get container status \"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee\": rpc error: code = NotFound desc = could not find container \"1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee\": container with ID starting with 1e3b796a0f76831d747bb3baaebee184b1c89ce31d0269b16f0a98bfd5cfb0ee not found: ID does not exist"
Nov 04 03:14:43.305223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.305148    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:14:43.354265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:43.354234    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4cbcf738-7b5dfbb998-l87xf"]
Nov 04 03:14:44.152579 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:44.152548    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:44.411695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:44.411576    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:44.556968 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:44.556927    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:44.981212 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:44.981185    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42f270fb-5da0-48c1-9011-64078fb7f5a2" path="/var/lib/kubelet/pods/42f270fb-5da0-48c1-9011-64078fb7f5a2/volumes"
Nov 04 03:14:48.727465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:48.727434    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:49.705698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:49.705664    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:52.366671 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:52.366628    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:52.540142 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:52.540111    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:54.744487 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:54.744458    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:54.914744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:54.914715    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:14:55.061647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:14:55.061611    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:00.097620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.097587    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:00.175492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.175455    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:00.176265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176237    6019 cpu_manager.go:414] "containerMap: removing container" podUID="42f270fb-5da0-48c1-9011-64078fb7f5a2" containerName="maas-api"
Nov 04 03:15:00.176265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176264    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f270fb-5da0-48c1-9011-64078fb7f5a2" containerName="maas-api"
Nov 04 03:15:00.176519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176294    6019 cpu_manager.go:414] "containerMap: removing container" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" containerName="test-curl-7027-5a1c78"
Nov 04 03:15:00.176519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176302    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" containerName="test-curl-7027-5a1c78"
Nov 04 03:15:00.176519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176409    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f270fb-5da0-48c1-9011-64078fb7f5a2" containerName="maas-api"
Nov 04 03:15:00.176519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.176426    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="081c7f5a-b033-4c7f-87a8-9254a5b75717" containerName="test-curl-7027-5a1c78"
Nov 04 03:15:00.179608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.179578    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:00.179776 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.179743    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.183712 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.183694    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.196299 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.196277    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:00.200354 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.200333    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.232498 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.232469    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slbq8\""
Nov 04 03:15:00.269971 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.269943    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:00.272215 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.272186    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:00.278249 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.278218    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9ns\" (UniqueName: \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.278377 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.278280    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.278452 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.278377    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.278515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.278461    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5264v\" (UniqueName: \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.322781 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.322747    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:00.379419 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379341    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9ns\" (UniqueName: \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.379419 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379409    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.379618 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379454    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.379618 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379497    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.379618 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379572    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5264v\" (UniqueName: \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.379765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.379637    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl7sw\" (UniqueName: \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.380358 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.380337    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.380475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.380435    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.441588 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.441555    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5264v\" (UniqueName: \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\") pod \"maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.441733 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.441565    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9ns\" (UniqueName: \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.481297 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.481264    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.481428 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.481349    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl7sw\" (UniqueName: \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.481846 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.481825    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.496027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.496003    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:00.502292 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.502264    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:00.543289 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.543226    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl7sw\" (UniqueName: \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\") pod \"maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.814524 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.814490    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:00.881856 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.881620    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:00.886737 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:15:00.885724    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3a9819_1b66_4ffd_8bb6_fc0b43d96e99.slice/crio-2329feb429dec77719291b7607b93ce58b6f73957000800aa29d3a24eb4c681e WatchSource:0}: Error finding container 2329feb429dec77719291b7607b93ce58b6f73957000800aa29d3a24eb4c681e: Status 404 returned error can't find the container with id 2329feb429dec77719291b7607b93ce58b6f73957000800aa29d3a24eb4c681e
Nov 04 03:15:00.929839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:00.929815    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:00.932006 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:15:00.931975    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1efd606a_4c47_42eb_88b8_7d8e64238fc3.slice/crio-9cda34592c93307d60e7caa376bf10e5810251e4054262bbc1e80be3f4881c44 WatchSource:0}: Error finding container 9cda34592c93307d60e7caa376bf10e5810251e4054262bbc1e80be3f4881c44: Status 404 returned error can't find the container with id 9cda34592c93307d60e7caa376bf10e5810251e4054262bbc1e80be3f4881c44
Nov 04 03:15:01.191458 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.191430    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:01.193713 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:15:01.193684    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821911de_8e59_4e75_81ba_d1c3caba37a2.slice/crio-f5443b589b85b57aa7d13d3b0d07158649a194f2d676dcbe833c4323abfd0d9c WatchSource:0}: Error finding container f5443b589b85b57aa7d13d3b0d07158649a194f2d676dcbe833c4323abfd0d9c: Status 404 returned error can't find the container with id f5443b589b85b57aa7d13d3b0d07158649a194f2d676dcbe833c4323abfd0d9c
Nov 04 03:15:01.233553 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.233525    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerStarted","Data":"b3bd38f70719859ba891f145e5ebb70a06fb77dfeca1251d8978dfb81284300c"}
Nov 04 03:15:01.233675 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.233595    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerStarted","Data":"2329feb429dec77719291b7607b93ce58b6f73957000800aa29d3a24eb4c681e"}
Nov 04 03:15:01.234782 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.234755    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerStarted","Data":"9cda34592c93307d60e7caa376bf10e5810251e4054262bbc1e80be3f4881c44"}
Nov 04 03:15:01.235819 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.235798    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerStarted","Data":"f5443b589b85b57aa7d13d3b0d07158649a194f2d676dcbe833c4323abfd0d9c"}
Nov 04 03:15:01.350390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:01.350322    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" podStartSLOduration=1.3503029610000001 podStartE2EDuration="1.350302961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:15:01.348019347 +0000 UTC m=+2456.489032760" watchObservedRunningTime="2026-11-04 03:15:01.350302961 +0000 UTC m=+2456.491316368"
Nov 04 03:15:02.242730 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.242687    6019 generic.go:356] "Generic (PLEG): container finished" podID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerID="b3bd38f70719859ba891f145e5ebb70a06fb77dfeca1251d8978dfb81284300c" exitCode=60
Nov 04 03:15:02.243202 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.242770    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerDied","Data":"b3bd38f70719859ba891f145e5ebb70a06fb77dfeca1251d8978dfb81284300c"}
Nov 04 03:15:02.243202 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.243040    6019 scope.go:122] "RemoveContainer" containerID="b3bd38f70719859ba891f145e5ebb70a06fb77dfeca1251d8978dfb81284300c"
Nov 04 03:15:02.244449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.244422    6019 generic.go:356] "Generic (PLEG): container finished" podID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerID="49e25970e1fcf0386b5909926ca6174010820f90706e0eaa2d07e4390ff73123" exitCode=60
Nov 04 03:15:02.244593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.244499    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerDied","Data":"49e25970e1fcf0386b5909926ca6174010820f90706e0eaa2d07e4390ff73123"}
Nov 04 03:15:02.244746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.244727    6019 scope.go:122] "RemoveContainer" containerID="49e25970e1fcf0386b5909926ca6174010820f90706e0eaa2d07e4390ff73123"
Nov 04 03:15:02.254287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.254270    6019 generic.go:356] "Generic (PLEG): container finished" podID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerID="be6ce5d93380481eb23e1c0c9193e02d2c625f9d7face7287f8b57aebab8a9a2" exitCode=60
Nov 04 03:15:02.254417 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.254394    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerDied","Data":"be6ce5d93380481eb23e1c0c9193e02d2c625f9d7face7287f8b57aebab8a9a2"}
Nov 04 03:15:02.254575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:02.254558    6019 scope.go:122] "RemoveContainer" containerID="be6ce5d93380481eb23e1c0c9193e02d2c625f9d7face7287f8b57aebab8a9a2"
Nov 04 03:15:03.263525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.263496    6019 generic.go:356] "Generic (PLEG): container finished" podID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerID="b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0" exitCode=60
Nov 04 03:15:03.263962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.263577    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerDied","Data":"b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0"}
Nov 04 03:15:03.263962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.263647    6019 scope.go:122] "RemoveContainer" containerID="49e25970e1fcf0386b5909926ca6174010820f90706e0eaa2d07e4390ff73123"
Nov 04 03:15:03.263962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.263929    6019 scope.go:122] "RemoveContainer" containerID="b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0"
Nov 04 03:15:03.264282 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:03.264259    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v_odh-ai-gateway-infra(1efd606a-4c47-42eb-88b8-7d8e64238fc3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3"
Nov 04 03:15:03.265706 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.265684    6019 generic.go:356] "Generic (PLEG): container finished" podID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerID="c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8" exitCode=60
Nov 04 03:15:03.265809 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.265715    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerDied","Data":"c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8"}
Nov 04 03:15:03.266025 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.266006    6019 scope.go:122] "RemoveContainer" containerID="c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8"
Nov 04 03:15:03.266318 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:03.266293    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v_odh-ai-gateway-infra(821911de-8e59-4e75-81ba-d1c3caba37a2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2"
Nov 04 03:15:03.267953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.267885    6019 generic.go:356] "Generic (PLEG): container finished" podID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerID="9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068" exitCode=60
Nov 04 03:15:03.267953 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.267925    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerDied","Data":"9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068"}
Nov 04 03:15:03.268272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.268184    6019 scope.go:122] "RemoveContainer" containerID="9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068"
Nov 04 03:15:03.268425 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:03.268406    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8_odh-ai-gateway-infra(fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99"
Nov 04 03:15:03.313437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.313417    6019 scope.go:122] "RemoveContainer" containerID="be6ce5d93380481eb23e1c0c9193e02d2c625f9d7face7287f8b57aebab8a9a2"
Nov 04 03:15:03.325261 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:03.325245    6019 scope.go:122] "RemoveContainer" containerID="b3bd38f70719859ba891f145e5ebb70a06fb77dfeca1251d8978dfb81284300c"
Nov 04 03:15:04.275463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:04.275424    6019 scope.go:122] "RemoveContainer" containerID="9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068"
Nov 04 03:15:04.275931 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:04.275684    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8_odh-ai-gateway-infra(fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99"
Nov 04 03:15:04.277115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:04.277092    6019 scope.go:122] "RemoveContainer" containerID="b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0"
Nov 04 03:15:04.277337 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:04.277312    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v_odh-ai-gateway-infra(1efd606a-4c47-42eb-88b8-7d8e64238fc3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3"
Nov 04 03:15:04.278593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:04.278574    6019 scope.go:122] "RemoveContainer" containerID="c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8"
Nov 04 03:15:04.278773 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:04.278754    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v_odh-ai-gateway-infra(821911de-8e59-4e75-81ba-d1c3caba37a2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2"
Nov 04 03:15:04.990303 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:04.990262    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:06.602512 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:06.602475    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:15:07.363864 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:07.363832    6019 scope.go:122] "RemoveContainer" containerID="915a8af52f0c74ab578b299f86c26c12848d5edacfb598d4ce9a78802175c1e4"
Nov 04 03:15:09.895847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:09.895757    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:13.975265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:13.975230    6019 scope.go:122] "RemoveContainer" containerID="9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068"
Nov 04 03:15:13.975669 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:13.975309    6019 scope.go:122] "RemoveContainer" containerID="c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8"
Nov 04 03:15:14.332870 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:14.332842    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerStarted","Data":"bbc455a5936af1d41941ca2afc911c96e41d47026cf77f24c2117159495a18d3"}
Nov 04 03:15:14.334769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:14.334744    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerStarted","Data":"330377969b03e56a913bc5e571ec1a162fcf6c41e7ff49aa41c5ecf7571b9f9b"}
Nov 04 03:15:14.563609 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:14.563497    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" podStartSLOduration=14.563482867 podStartE2EDuration="14.563482867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:15:14.561960945 +0000 UTC m=+2469.702974353" watchObservedRunningTime="2026-11-04 03:15:14.563482867 +0000 UTC m=+2469.704496274"
Nov 04 03:15:14.779316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:14.779285    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:14.978566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:14.978542    6019 scope.go:122] "RemoveContainer" containerID="b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0"
Nov 04 03:15:15.341879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.341841    6019 generic.go:356] "Generic (PLEG): container finished" podID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerID="bbc455a5936af1d41941ca2afc911c96e41d47026cf77f24c2117159495a18d3" exitCode=60
Nov 04 03:15:15.342015 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.341911    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerDied","Data":"bbc455a5936af1d41941ca2afc911c96e41d47026cf77f24c2117159495a18d3"}
Nov 04 03:15:15.342015 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.341967    6019 scope.go:122] "RemoveContainer" containerID="9730c10b49858edfc9491b414f765caaf619e2706e3e96f66198cfd53e2e0068"
Nov 04 03:15:15.342350 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.342331    6019 scope.go:122] "RemoveContainer" containerID="bbc455a5936af1d41941ca2afc911c96e41d47026cf77f24c2117159495a18d3"
Nov 04 03:15:15.342640 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:15.342618    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8_odh-ai-gateway-infra(fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99"
Nov 04 03:15:15.343827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.343806    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerStarted","Data":"c2ef870cf3a34031ddd3a5f9548a9da55dbdac9ed710c08bbc3cc29a679329bc"}
Nov 04 03:15:15.345521 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.345502    6019 generic.go:356] "Generic (PLEG): container finished" podID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerID="330377969b03e56a913bc5e571ec1a162fcf6c41e7ff49aa41c5ecf7571b9f9b" exitCode=60
Nov 04 03:15:15.345642 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.345572    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerDied","Data":"330377969b03e56a913bc5e571ec1a162fcf6c41e7ff49aa41c5ecf7571b9f9b"}
Nov 04 03:15:15.345890 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.345864    6019 scope.go:122] "RemoveContainer" containerID="330377969b03e56a913bc5e571ec1a162fcf6c41e7ff49aa41c5ecf7571b9f9b"
Nov 04 03:15:15.346137 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:15.346119    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v_odh-ai-gateway-infra(821911de-8e59-4e75-81ba-d1c3caba37a2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2"
Nov 04 03:15:15.354764 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.354747    6019 scope.go:122] "RemoveContainer" containerID="c45b58fe4fec3a9c2fe04420275d8a776625577db9eb36c10ced106c00cf59b8"
Nov 04 03:15:15.457198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.457134    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:15.571490 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.571456    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:15.585245 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:15.585195    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" podStartSLOduration=15.585181154 podStartE2EDuration="15.585181154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:15:15.582963488 +0000 UTC m=+2470.723976897" watchObservedRunningTime="2026-11-04 03:15:15.585181154 +0000 UTC m=+2470.726194585"
Nov 04 03:15:16.359948 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.359912    6019 generic.go:356] "Generic (PLEG): container finished" podID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerID="c2ef870cf3a34031ddd3a5f9548a9da55dbdac9ed710c08bbc3cc29a679329bc" exitCode=60
Nov 04 03:15:16.360514 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.360364    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerDied","Data":"c2ef870cf3a34031ddd3a5f9548a9da55dbdac9ed710c08bbc3cc29a679329bc"}
Nov 04 03:15:16.360695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.360682    6019 scope.go:122] "RemoveContainer" containerID="b44ce3e8d51f8f4eb28616e1c6011248d97b2466ee7e07690a655f6d1a99ded0"
Nov 04 03:15:16.361318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.361295    6019 scope.go:122] "RemoveContainer" containerID="c2ef870cf3a34031ddd3a5f9548a9da55dbdac9ed710c08bbc3cc29a679329bc"
Nov 04 03:15:16.362229 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:16.361797    6019 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v_odh-ai-gateway-infra(1efd606a-4c47-42eb-88b8-7d8e64238fc3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3"
Nov 04 03:15:16.512465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.512438    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:16.550061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.550033    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:16.578636 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.578606    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\") pod \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") "
Nov 04 03:15:16.578810 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.578794    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\" (UniqueName: \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\") pod \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\" (UID: \"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99\") "
Nov 04 03:15:16.578928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.578907    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca" pod "fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" (UID: "fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:15:16.579208 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.579195    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-maas-api-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:16.580957 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.580938    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v" pod "fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" (UID: "fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99"). InnerVolumeSpecName "kube-api-access-5264v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:15:16.592916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.592894    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:16.680005 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.679935    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\" (UniqueName: \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\") pod \"821911de-8e59-4e75-81ba-d1c3caba37a2\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") "
Nov 04 03:15:16.680160 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.680102    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\") pod \"821911de-8e59-4e75-81ba-d1c3caba37a2\" (UID: \"821911de-8e59-4e75-81ba-d1c3caba37a2\") "
Nov 04 03:15:16.680431 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.680399    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca" pod "821911de-8e59-4e75-81ba-d1c3caba37a2" (UID: "821911de-8e59-4e75-81ba-d1c3caba37a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:15:16.680431 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.680412    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5264v\" (UniqueName: \"kubernetes.io/projected/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99-kube-api-access-5264v\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:16.682150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.682129    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw" pod "821911de-8e59-4e75-81ba-d1c3caba37a2" (UID: "821911de-8e59-4e75-81ba-d1c3caba37a2"). InnerVolumeSpecName "kube-api-access-vl7sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:15:16.781773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.781742    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/821911de-8e59-4e75-81ba-d1c3caba37a2-maas-api-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:16.781773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:16.781769    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl7sw\" (UniqueName: \"kubernetes.io/projected/821911de-8e59-4e75-81ba-d1c3caba37a2-kube-api-access-vl7sw\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:17.373100 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.373043    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"
Nov 04 03:15:17.373586 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.373038    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8" event={"ID":"fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99","Type":"ContainerDied","Data":"2329feb429dec77719291b7607b93ce58b6f73957000800aa29d3a24eb4c681e"}
Nov 04 03:15:17.373586 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.373204    6019 scope.go:122] "RemoveContainer" containerID="bbc455a5936af1d41941ca2afc911c96e41d47026cf77f24c2117159495a18d3"
Nov 04 03:15:17.376785 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.376760    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"
Nov 04 03:15:17.376884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.376813    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v" event={"ID":"821911de-8e59-4e75-81ba-d1c3caba37a2","Type":"ContainerDied","Data":"f5443b589b85b57aa7d13d3b0d07158649a194f2d676dcbe833c4323abfd0d9c"}
Nov 04 03:15:17.383319 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.383297    6019 scope.go:122] "RemoveContainer" containerID="330377969b03e56a913bc5e571ec1a162fcf6c41e7ff49aa41c5ecf7571b9f9b"
Nov 04 03:15:17.506624 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.506598    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:17.535627 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.535598    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:17.539762 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.539739    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-203c35-29896035-7j85v"]
Nov 04 03:15:17.589952 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.589921    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\") pod \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") "
Nov 04 03:15:17.590118 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.590035    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\" (UniqueName: \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\") pod \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\" (UID: \"1efd606a-4c47-42eb-88b8-7d8e64238fc3\") "
Nov 04 03:15:17.590288 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.590268    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca" pod "1efd606a-4c47-42eb-88b8-7d8e64238fc3" (UID: "1efd606a-4c47-42eb-88b8-7d8e64238fc3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:15:17.590386 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.590371    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1efd606a-4c47-42eb-88b8-7d8e64238fc3-maas-api-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:17.592272 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.592247    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns" pod "1efd606a-4c47-42eb-88b8-7d8e64238fc3" (UID: "1efd606a-4c47-42eb-88b8-7d8e64238fc3"). InnerVolumeSpecName "kube-api-access-cl9ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:15:17.691761 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.691678    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl9ns\" (UniqueName: \"kubernetes.io/projected/1efd606a-4c47-42eb-88b8-7d8e64238fc3-kube-api-access-cl9ns\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:15:17.766031 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.765995    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:17.768532 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:17.768506    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-db5ef5-29896035-488l8"]
Nov 04 03:15:18.384583 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.384550    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"
Nov 04 03:15:18.384583 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.384561    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v" event={"ID":"1efd606a-4c47-42eb-88b8-7d8e64238fc3","Type":"ContainerDied","Data":"9cda34592c93307d60e7caa376bf10e5810251e4054262bbc1e80be3f4881c44"}
Nov 04 03:15:18.385168 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.384626    6019 scope.go:122] "RemoveContainer" containerID="c2ef870cf3a34031ddd3a5f9548a9da55dbdac9ed710c08bbc3cc29a679329bc"
Nov 04 03:15:18.548620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.548591    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:18.555022 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.554993    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-35df3e-29896035-hps4v"]
Nov 04 03:15:18.981757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.981722    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" path="/var/lib/kubelet/pods/1efd606a-4c47-42eb-88b8-7d8e64238fc3/volumes"
Nov 04 03:15:18.982192 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.982178    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" path="/var/lib/kubelet/pods/821911de-8e59-4e75-81ba-d1c3caba37a2/volumes"
Nov 04 03:15:18.982537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:18.982526    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" path="/var/lib/kubelet/pods/fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99/volumes"
Nov 04 03:15:19.563048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:19.563015    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:19.724973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:19.724940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:19.879059 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:19.878973    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:24.686664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:24.686637    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:29.603029 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:29.602989    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:34.496189 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:34.496156    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:36.603066 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:15:36.603028    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:15:36.814032 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:36.814000    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:39.442807 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:39.442767    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:39.990041 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:39.990007    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:42.152142 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:42.152107    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:44.288189 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:44.288151    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:49.186308 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:49.186270    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:51.539822 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:51.539787    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:51.683745 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:51.683708    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:54.095132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:54.095101    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:15:59.000224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:15:59.000191    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:01.491711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:01.491675    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:03.976665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:03.976628    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:06.603447 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:06.603409    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:16:09.088547 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:09.088497    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:11.260287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260256    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7027-cdfc76"]
Nov 04 03:16:11.260714 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260701    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260715    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260727    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260732    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260743    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260749    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260758    6019 cpu_manager.go:414] "containerMap: removing container" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260765 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260762    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260778    6019 cpu_manager.go:414] "containerMap: removing container" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260783    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260791    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260796    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260804    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260808    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260820    6019 cpu_manager.go:414] "containerMap: removing container" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260824    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260887    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260896    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260902    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260907    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260915    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="1efd606a-4c47-42eb-88b8-7d8e64238fc3" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260920    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260926    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260932    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd3a9819-1b66-4ffd-8bb6-fc0b43d96e99" containerName="cleanup"
Nov 04 03:16:11.260980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.260941    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:16:11.265274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.265253    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:11.318625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.318600    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7027-cdfc76"]
Nov 04 03:16:11.365783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.365761    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hzl5n\""
Nov 04 03:16:11.394492 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.394458    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwhg\" (UniqueName: \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\") pod \"test-curl-7027-cdfc76\" (UID: \"03438791-5762-47bc-9304-03da49f041b4\") " pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:11.495608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.495572    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwhg\" (UniqueName: \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\") pod \"test-curl-7027-cdfc76\" (UID: \"03438791-5762-47bc-9304-03da49f041b4\") " pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:11.556941 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.556865    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwhg\" (UniqueName: \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\") pod \"test-curl-7027-cdfc76\" (UID: \"03438791-5762-47bc-9304-03da49f041b4\") " pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:11.576910 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.576882    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:11.952484 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:11.952458    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7027-cdfc76"]
Nov 04 03:16:11.955337 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:16:11.955303    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03438791_5762_47bc_9304_03da49f041b4.slice/crio-848014d1bde464ab67e465e4a9013ea0fa776a1944b74d3045937aeafdfacb1a WatchSource:0}: Error finding container 848014d1bde464ab67e465e4a9013ea0fa776a1944b74d3045937aeafdfacb1a: Status 404 returned error can't find the container with id 848014d1bde464ab67e465e4a9013ea0fa776a1944b74d3045937aeafdfacb1a
Nov 04 03:16:12.697998 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:12.697964    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7027-cdfc76" event={"ID":"03438791-5762-47bc-9304-03da49f041b4","Type":"ContainerStarted","Data":"f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89"}
Nov 04 03:16:12.698423 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:12.698006    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7027-cdfc76" event={"ID":"03438791-5762-47bc-9304-03da49f041b4","Type":"ContainerStarted","Data":"848014d1bde464ab67e465e4a9013ea0fa776a1944b74d3045937aeafdfacb1a"}
Nov 04 03:16:12.811665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:12.811612    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7027-cdfc76" podStartSLOduration=1.811597221 podStartE2EDuration="1.811597221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:12.810677988 +0000 UTC m=+2527.951691396" watchObservedRunningTime="2026-11-04 03:16:12.811597221 +0000 UTC m=+2527.952610630"
Nov 04 03:16:13.995398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:13.995370    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:14.263724 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:14.263694    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7027-cdfc76"]
Nov 04 03:16:14.270180 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:14.270155    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7027-cdfc76"]
Nov 04 03:16:14.709663 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:14.709577    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7027-cdfc76" podUID="03438791-5762-47bc-9304-03da49f041b4" containerName="test-curl-7027-cdfc76" containerID="cri-o://f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89" gracePeriod=2
Nov 04 03:16:14.760630 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:14.760595    6019 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7027-cdfc76\" is forbidden: User \"system:node:ip-10-0-128-201.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-201.us-west-1.compute.internal' and this object" podUID="03438791-5762-47bc-9304-03da49f041b4" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:14.787713 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:14.787680    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:15.028585 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:15.028554    6019 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7027-cdfc76\" is forbidden: User \"system:node:ip-10-0-128-201.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-201.us-west-1.compute.internal' and this object" podUID="03438791-5762-47bc-9304-03da49f041b4" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:16.957141 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:16.957116    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:17.057451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.057424    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\" (UniqueName: \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\") pod \"03438791-5762-47bc-9304-03da49f041b4\" (UID: \"03438791-5762-47bc-9304-03da49f041b4\") "
Nov 04 03:16:17.059676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.059647    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg" pod "03438791-5762-47bc-9304-03da49f041b4" (UID: "03438791-5762-47bc-9304-03da49f041b4"). InnerVolumeSpecName "kube-api-access-hdwhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:17.159247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.159213    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdwhg\" (UniqueName: \"kubernetes.io/projected/03438791-5762-47bc-9304-03da49f041b4-kube-api-access-hdwhg\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:17.726912 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.726879    6019 generic.go:356] "Generic (PLEG): container finished" podID="03438791-5762-47bc-9304-03da49f041b4" containerID="f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89" exitCode=137
Nov 04 03:16:17.727107 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.726928    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7027-cdfc76"
Nov 04 03:16:17.727107 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.726946    6019 scope.go:122] "RemoveContainer" containerID="f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89"
Nov 04 03:16:17.737845 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.737831    6019 scope.go:122] "RemoveContainer" containerID="f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89"
Nov 04 03:16:17.738164 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:17.738137    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89\": container with ID starting with f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89 not found: ID does not exist" containerID="f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89"
Nov 04 03:16:17.738224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:17.738162    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89"} err="failed to get container status \"f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89\": rpc error: code = NotFound desc = could not find container \"f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89\": container with ID starting with f76022255ba8991c3c4f573439ac2d8d4a8a415a67c941a57571b3fb25adee89 not found: ID does not exist"
Nov 04 03:16:18.981494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:18.981462    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03438791-5762-47bc-9304-03da49f041b4" path="/var/lib/kubelet/pods/03438791-5762-47bc-9304-03da49f041b4/volumes"
Nov 04 03:16:18.987924 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:18.987900    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:19.741969 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:19.741936    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:16:19.838187 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:19.838139    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:22.156094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:22.156051    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:22.299757 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:22.299724    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:23.883217 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:23.883186    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:28.781301 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:28.781260    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:29.539926 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:29.539892    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:30.321420 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.321383    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 03:16:30.321854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.321562    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy" containerID="cri-o://cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8" gracePeriod=30
Nov 04 03:16:30.480105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.480041    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 03:16:30.480319 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.480289    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" containerID="cri-o://8e5619dc6b0e5e07f8e5d9317543a3b3a9b35f604f0fed652831cb33bc62df4c" gracePeriod=30
Nov 04 03:16:30.493542 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.493503    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:16:30.493815 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.493787    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy" containerID="cri-o://4ed1a2c2ba843f60b97629cf0f6d26e8043189304a283bbb0842dbd145924060" gracePeriod=30
Nov 04 03:16:30.496000 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.495975    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 03:16:30.496216 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:30.496195    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" containerID="cri-o://ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2" gracePeriod=30
Nov 04 03:16:31.818594 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:31.818555    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-xrlzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:16:31.818978 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:31.818616    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:16:32.609684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:32.609644    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:16:32.609887 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:32.609725    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:16:33.669681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:33.669642    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:34.763654 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:34.763612    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" podUID="090014a4-3291-44d8-9784-e151bfd3cb0d" containerName="main" containerID="cri-o://5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011" gracePeriod=45
Nov 04 03:16:35.026369 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.026303    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:16:35.058293 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058257    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058338    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\" (UniqueName: \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058371    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058407    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058440    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058640 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058504    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\" (UniqueName: \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\") pod \"090014a4-3291-44d8-9784-e151bfd3cb0d\" (UID: \"090014a4-3291-44d8-9784-e151bfd3cb0d\") "
Nov 04 03:16:35.058640 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058588    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.058762 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058685    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.058762 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058715    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.059009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.058986    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.059009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.059009    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.059243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.059025    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.061055 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.061027    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.061183 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.061155    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "kube-api-access-hxzk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:35.061229 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.061196    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs" pod "090014a4-3291-44d8-9784-e151bfd3cb0d" (UID: "090014a4-3291-44d8-9784-e151bfd3cb0d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:16:35.160566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.160518    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/090014a4-3291-44d8-9784-e151bfd3cb0d-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.160566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.160549    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/090014a4-3291-44d8-9784-e151bfd3cb0d-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.160566 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.160561    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxzk4\" (UniqueName: \"kubernetes.io/projected/090014a4-3291-44d8-9784-e151bfd3cb0d-kube-api-access-hxzk4\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.671334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.671261    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 03:16:35.736957 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.736913    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 03:16:35.767724 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767679    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767738    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767777    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767805    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767837    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767863    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767918    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.767990    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768044    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768086    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768117    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768176    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768223    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768263    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768313    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.768376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768350    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\") pod \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\" (UID: \"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771\") "
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768402    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.768433    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\") pod \"cfda9917-4319-4230-89a0-7624e0236375\" (UID: \"cfda9917-4319-4230-89a0-7624e0236375\") "
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.769478    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.769815    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.770064    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.777199    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.777493    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.778253    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.780650    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.780850    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781014    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781211    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781447    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781465    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781479    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781494    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfda9917-4319-4230-89a0-7624e0236375-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781509    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781523    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781536    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781551    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781566    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.781578    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.787252    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.788298    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "kube-api-access-mggkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.790757    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.792975    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:35.797281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.793131    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "kube-api-access-mg7fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:35.799009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.797997    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:16:35.809684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.809651    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:16:35.810290 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.810261    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy" pod "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" (UID: "dc093e4d-e9ca-4bc2-8bf3-9c95dc748771"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.810682 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.810497    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy" pod "cfda9917-4319-4230-89a0-7624e0236375" (UID: "cfda9917-4319-4230-89a0-7624e0236375"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.839235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.839197    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:35.840753 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.840725    6019 generic.go:356] "Generic (PLEG): container finished" podID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerID="ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2" exitCode=0
Nov 04 03:16:35.841003 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.840863    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"
Nov 04 03:16:35.841003 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.840876    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" event={"ID":"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771","Type":"ContainerDied","Data":"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"}
Nov 04 03:16:35.841003 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.840992    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6" event={"ID":"dc093e4d-e9ca-4bc2-8bf3-9c95dc748771","Type":"ContainerDied","Data":"ab74886c2facbf8d5ee8ea84417c4bdceec08d6467784263bd424a05ccd98be2"}
Nov 04 03:16:35.843020 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.841034    6019 scope.go:122] "RemoveContainer" containerID="ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"
Nov 04 03:16:35.843262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.843130    6019 generic.go:356] "Generic (PLEG): container finished" podID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerID="8e5619dc6b0e5e07f8e5d9317543a3b3a9b35f604f0fed652831cb33bc62df4c" exitCode=0
Nov 04 03:16:35.843399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.843368    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" event={"ID":"556dc3ec-4888-4b5d-909b-6a5023593b55","Type":"ContainerDied","Data":"8e5619dc6b0e5e07f8e5d9317543a3b3a9b35f604f0fed652831cb33bc62df4c"}
Nov 04 03:16:35.843485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.843462    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk" event={"ID":"556dc3ec-4888-4b5d-909b-6a5023593b55","Type":"ContainerDied","Data":"ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22"}
Nov 04 03:16:35.843758 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.843506    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec1a1b064fffae019dba9b3f32b3dd06376487295973629ea9a1c235c032dd22"
Nov 04 03:16:35.846019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.845966    6019 generic.go:356] "Generic (PLEG): container finished" podID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerID="4ed1a2c2ba843f60b97629cf0f6d26e8043189304a283bbb0842dbd145924060" exitCode=0
Nov 04 03:16:35.846226 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.846199    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" event={"ID":"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e","Type":"ContainerDied","Data":"4ed1a2c2ba843f60b97629cf0f6d26e8043189304a283bbb0842dbd145924060"}
Nov 04 03:16:35.846314 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.846285    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw" event={"ID":"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e","Type":"ContainerDied","Data":"48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b"}
Nov 04 03:16:35.846375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.846319    6019 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48a03ed8a695f2aceb4848f787ef86600fa26d1ce9ed4c057cc79466eed4231b"
Nov 04 03:16:35.863046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.859769    6019 generic.go:356] "Generic (PLEG): container finished" podID="090014a4-3291-44d8-9784-e151bfd3cb0d" containerID="5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011" exitCode=0
Nov 04 03:16:35.863046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.859923    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"
Nov 04 03:16:35.863046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.862266    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" event={"ID":"090014a4-3291-44d8-9784-e151bfd3cb0d","Type":"ContainerDied","Data":"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"}
Nov 04 03:16:35.863046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.862310    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5" event={"ID":"090014a4-3291-44d8-9784-e151bfd3cb0d","Type":"ContainerDied","Data":"ed8b7e9c4c2d5c8666161c9323853d2233a2953a253d2beffdcc2b53b922c445"}
Nov 04 03:16:35.863046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.862884    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 03:16:35.863475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.863354    6019 generic.go:356] "Generic (PLEG): container finished" podID="cfda9917-4319-4230-89a0-7624e0236375" containerID="cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8" exitCode=0
Nov 04 03:16:35.863475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.863397    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" event={"ID":"cfda9917-4319-4230-89a0-7624e0236375","Type":"ContainerDied","Data":"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"}
Nov 04 03:16:35.863475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.863436    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt" event={"ID":"cfda9917-4319-4230-89a0-7624e0236375","Type":"ContainerDied","Data":"1f706acada5d2eece4ee20f5cf8a2dac2864f4af70d68d57a7f41614ce079612"}
Nov 04 03:16:35.863475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.863444    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"
Nov 04 03:16:35.867915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.867888    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:16:35.871966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.871934    6019 scope.go:122] "RemoveContainer" containerID="ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"
Nov 04 03:16:35.872310 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:35.872287    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2\": container with ID starting with ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2 not found: ID does not exist" containerID="ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"
Nov 04 03:16:35.872437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.872322    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2"} err="failed to get container status \"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2\": rpc error: code = NotFound desc = could not find container \"ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2\": container with ID starting with ec9a668b5e8924d1d6d97e87dafcefa770b8cec6fdf4687b5d8c8693a2ef22a2 not found: ID does not exist"
Nov 04 03:16:35.872437 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.872340    6019 scope.go:122] "RemoveContainer" containerID="5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"
Nov 04 03:16:35.881885 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.881863    6019 scope.go:122] "RemoveContainer" containerID="5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"
Nov 04 03:16:35.882247 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:35.882225    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011\": container with ID starting with 5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011 not found: ID does not exist" containerID="5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"
Nov 04 03:16:35.882328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882256    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011"} err="failed to get container status \"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011\": rpc error: code = NotFound desc = could not find container \"5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011\": container with ID starting with 5146fe50743353add4e4227938d5b9c1fb3f5bf34767cb7cc5d1b93321d17011 not found: ID does not exist"
Nov 04 03:16:35.882328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882277    6019 scope.go:122] "RemoveContainer" containerID="cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"
Nov 04 03:16:35.882447 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882418    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfda9917-4319-4230-89a0-7624e0236375-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882447 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882437    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882452    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mggkm\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-kube-api-access-mggkm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882467    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882479    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882493    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg7fz\" (UniqueName: \"kubernetes.io/projected/cfda9917-4319-4230-89a0-7624e0236375-kube-api-access-mg7fz\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882506    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfda9917-4319-4230-89a0-7624e0236375-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.882554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.882514    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:35.891854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.891822    6019 scope.go:122] "RemoveContainer" containerID="cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"
Nov 04 03:16:35.892193 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:35.892169    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8\": container with ID starting with cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8 not found: ID does not exist" containerID="cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"
Nov 04 03:16:35.892286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.892200    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8"} err="failed to get container status \"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8\": rpc error: code = NotFound desc = could not find container \"cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8\": container with ID starting with cb2bbbe3806af73da435b5ebda284215728d68a7b0284167178e04d1cb67caf8 not found: ID does not exist"
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.982865    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.982930    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.982959    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.982987    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983053    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983137    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983178    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983206    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983270    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983299    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983341    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983372    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983412    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983444    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983481    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\") pod \"556dc3ec-4888-4b5d-909b-6a5023593b55\" (UID: \"556dc3ec-4888-4b5d-909b-6a5023593b55\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983541    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983575    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.984375 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.983661    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\") pod \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\" (UID: \"41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e\") "
Nov 04 03:16:35.986124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.985842    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.987012 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.986863    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:16:35.987138 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.987007    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.987497 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.987348    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.987758 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.987605    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.988006 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.987863    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.988299 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.988141    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.988668 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.988482    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:16:35.989010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.988782    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:35.989457 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:35.989215    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:36.005615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.004415    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:36.005615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.004486    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:36.005615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.004676    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:36.005615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.004698    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "kube-api-access-6lk5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:36.005615 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.004729    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:16:36.008105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.008018    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:36.008105 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.008094    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj" pod "556dc3ec-4888-4b5d-909b-6a5023593b55" (UID: "556dc3ec-4888-4b5d-909b-6a5023593b55"). InnerVolumeSpecName "kube-api-access-dg4hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:36.008328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.008143    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo" pod "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" (UID: "41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084773    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084812    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084829    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg4hj\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-kube-api-access-dg4hj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084845    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084860    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084872    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084884    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084898    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084912    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084925    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084939    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084952    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lk5g\" (UniqueName: \"kubernetes.io/projected/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-kube-api-access-6lk5g\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084965    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084979    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.084991    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/556dc3ec-4888-4b5d-909b-6a5023593b55-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.085004    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.085012    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.085114 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.085023    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/556dc3ec-4888-4b5d-909b-6a5023593b55-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:36.311956 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.311918    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 03:16:36.314390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.314366    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt"]
Nov 04 03:16:36.323094 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.323041    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:36.423315 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.423280    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 03:16:36.430095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.430041    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6"]
Nov 04 03:16:36.603879 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:36.603794    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:16:36.661701 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.661669    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:16:36.664483 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.664452    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4cbcf738/e2e-lifecycle-model-4cbcf738-kserve-5f464c755c-vgkb5"]
Nov 04 03:16:36.871756 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.871672    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"
Nov 04 03:16:36.872243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.871881    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"
Nov 04 03:16:36.982296 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.982254    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="090014a4-3291-44d8-9784-e151bfd3cb0d" path="/var/lib/kubelet/pods/090014a4-3291-44d8-9784-e151bfd3cb0d/volumes"
Nov 04 03:16:36.982736 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.982720    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfda9917-4319-4230-89a0-7624e0236375" path="/var/lib/kubelet/pods/cfda9917-4319-4230-89a0-7624e0236375/volumes"
Nov 04 03:16:36.983225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:36.983203    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" path="/var/lib/kubelet/pods/dc093e4d-e9ca-4bc2-8bf3-9c95dc748771/volumes"
Nov 04 03:16:37.038674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:37.038639    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 03:16:37.087995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:37.087957    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk"]
Nov 04 03:16:37.155904 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:37.155814    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:16:37.161860 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:37.161827    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw"]
Nov 04 03:16:38.582694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:38.582661    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:38.980800 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:38.980722    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" path="/var/lib/kubelet/pods/41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e/volumes"
Nov 04 03:16:38.981218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:38.981203    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" path="/var/lib/kubelet/pods/556dc3ec-4888-4b5d-909b-6a5023593b55/volumes"
Nov 04 03:16:40.354938 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:40.354859    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:42.499569 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:42.499510    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:42.700357 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:42.700318    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:43.625218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:43.625181    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:44.969847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:44.969812    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:45.114596 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:45.114559    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:48.276733 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:48.276699    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:48.691593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:48.691508    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:50.870714 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:50.870657    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" podUID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" containerName="main" containerID="cri-o://57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5" gracePeriod=45
Nov 04 03:16:51.124212 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.124155    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:16:51.237309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237277    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237331    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237370    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237421    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\" (UniqueName: \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237474 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237460    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237513    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\") pod \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\" (UID: \"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89\") "
Nov 04 03:16:51.237681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237558    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:51.237681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237670    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:51.237842 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237724    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:51.238009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.237989    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.238102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.238018    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.238102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.238031    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.239722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.239685    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:16:51.239849 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.239822    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "kube-api-access-6g4fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:16:51.240139 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.239849    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm" pod "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" (UID: "bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:16:51.339522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.339493    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g4fg\" (UniqueName: \"kubernetes.io/projected/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-kube-api-access-6g4fg\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.339522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.339519    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.339522 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.339529    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:16:51.950472 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.950429    6019 generic.go:356] "Generic (PLEG): container finished" podID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" containerID="57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5" exitCode=0
Nov 04 03:16:51.950992 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.950489    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"
Nov 04 03:16:51.950992 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.950519    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" event={"ID":"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89","Type":"ContainerDied","Data":"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"}
Nov 04 03:16:51.950992 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.950568    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8" event={"ID":"bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89","Type":"ContainerDied","Data":"1204bda02b6e461d1dc7c80028092400b2ec8b4c6d5b302cb1e7b7a6af2e75fb"}
Nov 04 03:16:51.950992 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.950588    6019 scope.go:122] "RemoveContainer" containerID="57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"
Nov 04 03:16:51.960338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.960321    6019 scope.go:122] "RemoveContainer" containerID="57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"
Nov 04 03:16:51.960599 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:16:51.960578    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5\": container with ID starting with 57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5 not found: ID does not exist" containerID="57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"
Nov 04 03:16:51.960647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:51.960608    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5"} err="failed to get container status \"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5\": rpc error: code = NotFound desc = could not find container \"57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5\": container with ID starting with 57095d8cc4d103ab1e8378cddd9dc213d6fb2eda18fb532a56bed25cc8d4d2f5 not found: ID does not exist"
Nov 04 03:16:52.113977 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:52.113946    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:16:52.163254 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:52.163220    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-095081/test-model-095081-kserve-64c84c89d9-qznj8"]
Nov 04 03:16:52.979857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:52.979822    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" path="/var/lib/kubelet/pods/bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89/volumes"
Nov 04 03:16:53.583089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:53.583039    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:57.061326 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:57.061285    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:57.291032 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:57.290996    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:57.869843 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:57.869786    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:58.019710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:58.019669    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:16:58.167036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:16:58.166917    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:00.121749 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:00.121711    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:03.201806 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:03.201774    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:03.344643 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:03.344603    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:03.598622 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:03.598579    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:04.860167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:04.860116    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:05.458363 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:05.458332    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:05.612099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:05.608843    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.272575 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:06.272540    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.602682 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:17:06.602591    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:17:07.407941 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:07.407911    6019 scope.go:122] "RemoveContainer" containerID="4ed1a2c2ba843f60b97629cf0f6d26e8043189304a283bbb0842dbd145924060"
Nov 04 03:17:07.418309 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:07.418291    6019 scope.go:122] "RemoveContainer" containerID="8e5619dc6b0e5e07f8e5d9317543a3b3a9b35f604f0fed652831cb33bc62df4c"
Nov 04 03:17:09.755430 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:09.755396    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:10.604860 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:10.604823    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:12.457543 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:12.457502    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:12.601672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:12.601624    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:12.747218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:12.747127    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:15.191767 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:15.191721    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:17.076767 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:17.076732    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:18.713535 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:18.713499    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:21.987106 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:21.987056    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:24.896334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:24.896287    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:25.040371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:25.040326    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:26.487243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:26.487206    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:27.037293 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:27.037256    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:27.185536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:27.185501    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:27.499933 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:27.499900    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:29.290353 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:29.290319    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 03:17:29.290789 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:29.290523    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy" containerID="cri-o://9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896" gracePeriod=30
Nov 04 03:17:32.397056 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:32.397015    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:32.460874 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:32.460836    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 03:17:32.582059 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:32.582006    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:34.544559 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.544537    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 03:17:34.684737 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684667    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684737 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684703    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684737 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684737    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684763    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684795    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684857    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684934    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.684973 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.684969    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.685287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685049    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\") pod \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\" (UID: \"8b2c92fc-84e0-4959-9eb2-59fcc6572539\") "
Nov 04 03:17:34.685287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685121    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:34.685287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685137    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:34.685287 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685218    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:34.685520 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685501    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.685580 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685528    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.685580 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685543    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.685657 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685616    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:34.685700 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.685658    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:17:34.687190 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.687160    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:17:34.687355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.687272    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:34.687355 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.687295    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "kube-api-access-vx8wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:17:34.687507 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.687489    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token" pod "8b2c92fc-84e0-4959-9eb2-59fcc6572539" (UID: "8b2c92fc-84e0-4959-9eb2-59fcc6572539"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:17:34.786479 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786447    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.786479 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786473    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.786479 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786485    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx8wl\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-kube-api-access-vx8wl\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.786719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786494    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b2c92fc-84e0-4959-9eb2-59fcc6572539-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.786719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786502    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:34.786719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:34.786511    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b2c92fc-84e0-4959-9eb2-59fcc6572539-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:35.175936 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.175901    6019 generic.go:356] "Generic (PLEG): container finished" podID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerID="9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896" exitCode=0
Nov 04 03:17:35.176158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.175954    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"
Nov 04 03:17:35.176158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.175980    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" event={"ID":"8b2c92fc-84e0-4959-9eb2-59fcc6572539","Type":"ContainerDied","Data":"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"}
Nov 04 03:17:35.176158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.176032    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r" event={"ID":"8b2c92fc-84e0-4959-9eb2-59fcc6572539","Type":"ContainerDied","Data":"5c5bbc1effe609b5cfb30fcfe8f1fd50f5928d2eb998f2394897faf25cae27aa"}
Nov 04 03:17:35.176158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.176053    6019 scope.go:122] "RemoveContainer" containerID="9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"
Nov 04 03:17:35.185271 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.185257    6019 scope.go:122] "RemoveContainer" containerID="9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"
Nov 04 03:17:35.185557 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:17:35.185539    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896\": container with ID starting with 9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896 not found: ID does not exist" containerID="9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"
Nov 04 03:17:35.185619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.185565    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896"} err="failed to get container status \"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896\": rpc error: code = NotFound desc = could not find container \"9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896\": container with ID starting with 9e59804d9e125c4d002d2fd63e9e009ec803a349c1a8dc158cf0bc86fc20a896 not found: ID does not exist"
Nov 04 03:17:35.340824 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.340794    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 03:17:35.390711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:35.390676    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r"]
Nov 04 03:17:36.616339 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:17:36.616295    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:17:36.661731 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:36.661687    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:36.980787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:36.980709    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" path="/var/lib/kubelet/pods/8b2c92fc-84e0-4959-9eb2-59fcc6572539/volumes"
Nov 04 03:17:38.479335 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:38.479297    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:38.808313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:38.808275    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.709247 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:40.709210    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:42.663865 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:42.663820    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:42.846693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:42.846660    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:42.992879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:42.992770    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:43.436222 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:43.436180    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:44.308934 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:44.308899    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:44.834411 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:44.834380    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:45.397985 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:45.397944    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:45.640608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:45.640565    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:45.949861 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:45.949832    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:46.091126 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:46.091088    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.482791 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.482754    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" containerID="cri-o://b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155" gracePeriod=45
Nov 04 03:17:47.736470 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.736407    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 03:17:47.831854 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.831827    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\" (UniqueName: \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.831990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.831874    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.831990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.831914    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.831990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.831961    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.832177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832005    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\" (UniqueName: \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.832177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832029    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\") pod \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\" (UID: \"0405bd41-0c36-43a5-9aef-41f06a6bc22b\") "
Nov 04 03:17:47.832277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832239    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:47.832384 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832349    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:47.832442 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832362    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:47.832442 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.832417    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:47.834328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.834304    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:17:47.834427 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.834382    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:17:47.834563 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.834536    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f" pod "0405bd41-0c36-43a5-9aef-41f06a6bc22b" (UID: "0405bd41-0c36-43a5-9aef-41f06a6bc22b"). InnerVolumeSpecName "kube-api-access-ttp7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:17:47.933468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.933437    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttp7f\" (UniqueName: \"kubernetes.io/projected/0405bd41-0c36-43a5-9aef-41f06a6bc22b-kube-api-access-ttp7f\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:47.933468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.933466    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:47.933468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.933476    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0405bd41-0c36-43a5-9aef-41f06a6bc22b-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:47.933703 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.933484    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:47.933703 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:47.933492    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0405bd41-0c36-43a5-9aef-41f06a6bc22b-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:48.245686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.245649    6019 generic.go:356] "Generic (PLEG): container finished" podID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerID="b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155" exitCode=0
Nov 04 03:17:48.245902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.245695    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"
Nov 04 03:17:48.245902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.245744    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" event={"ID":"0405bd41-0c36-43a5-9aef-41f06a6bc22b","Type":"ContainerDied","Data":"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"}
Nov 04 03:17:48.245902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.245797    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" event={"ID":"0405bd41-0c36-43a5-9aef-41f06a6bc22b","Type":"ContainerDied","Data":"eb74f3dce463113d6c1d357587fe79b75664069e7ea24969def97487af2caa79"}
Nov 04 03:17:48.245902 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.245820    6019 scope.go:122] "RemoveContainer" containerID="b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"
Nov 04 03:17:48.256235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.256215    6019 scope.go:122] "RemoveContainer" containerID="b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"
Nov 04 03:17:48.256502 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:17:48.256483    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155\": container with ID starting with b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155 not found: ID does not exist" containerID="b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"
Nov 04 03:17:48.256568 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.256509    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155"} err="failed to get container status \"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155\": rpc error: code = NotFound desc = could not find container \"b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155\": container with ID starting with b598f8f1c37619b8db021b331ccd3a9fbc62a1915af1b4f706cb37e092f61155 not found: ID does not exist"
Nov 04 03:17:48.409052 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.409020    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 03:17:48.413946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.413915    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs"]
Nov 04 03:17:48.676666 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.676635    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:48.696547 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.696519    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a0bcbc/distinct-2-w2-a0bcbc-kserve-86d675dfc-fx5vs" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:17:48.980921 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:48.980849    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" path="/var/lib/kubelet/pods/0405bd41-0c36-43a5-9aef-41f06a6bc22b/volumes"
Nov 04 03:17:51.632344 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:51.632306    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:51.892840 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:51.892757    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:53.004337 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:53.004298    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 03:17:53.123627 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:53.123590    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 03:17:53.372853 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:53.372775    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:53.516091 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:53.516035    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:53.942468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:53.942439    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:57.573493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:57.573450    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:57.958326 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:57.958238    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:58.005225 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.005172    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" podUID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" containerName="payload-processing" containerID="cri-o://b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6" gracePeriod=25
Nov 04 03:17:58.123853 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.123821    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" podUID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" containerName="payload-pre-processing" containerID="cri-o://1dcd8be61b57327bdf4ed70d6f2d4f21bad491e1caf80917d03b08fe51c00bc5" gracePeriod=25
Nov 04 03:17:58.257978 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.257952    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 03:17:58.297922 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.297888    6019 generic.go:356] "Generic (PLEG): container finished" podID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" containerID="b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6" exitCode=0
Nov 04 03:17:58.298048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.297932    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"
Nov 04 03:17:58.298048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.297967    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" event={"ID":"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f","Type":"ContainerDied","Data":"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"}
Nov 04 03:17:58.298159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.298051    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8" event={"ID":"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f","Type":"ContainerDied","Data":"1ff684a9ba13638ed5c302332ec6787e38b1cde238beac80c86b748d32a078b5"}
Nov 04 03:17:58.298159 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.298108    6019 scope.go:122] "RemoveContainer" containerID="b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"
Nov 04 03:17:58.299904 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.299880    6019 generic.go:356] "Generic (PLEG): container finished" podID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" containerID="1dcd8be61b57327bdf4ed70d6f2d4f21bad491e1caf80917d03b08fe51c00bc5" exitCode=0
Nov 04 03:17:58.300027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.299935    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" event={"ID":"ac4af128-f2dd-4930-bb1f-baf8339f2bd4","Type":"ContainerDied","Data":"1dcd8be61b57327bdf4ed70d6f2d4f21bad491e1caf80917d03b08fe51c00bc5"}
Nov 04 03:17:58.309008 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.308993    6019 scope.go:122] "RemoveContainer" containerID="b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"
Nov 04 03:17:58.309348 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:17:58.309322    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6\": container with ID starting with b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6 not found: ID does not exist" containerID="b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"
Nov 04 03:17:58.309422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.309362    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6"} err="failed to get container status \"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6\": rpc error: code = NotFound desc = could not find container \"b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6\": container with ID starting with b205fc53b9327106d5d6e3326818712958c55955439d2be20a0130de00cbfde6 not found: ID does not exist"
Nov 04 03:17:58.332625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.332600    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\") pod \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") "
Nov 04 03:17:58.332754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.332677    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\" (UniqueName: \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\") pod \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\" (UID: \"40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f\") "
Nov 04 03:17:58.332990 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.332967    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume" pod "40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" (UID: "40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:17:58.335121 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.335048    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk" pod "40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" (UID: "40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f"). InnerVolumeSpecName "kube-api-access-bw9kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:17:58.369540 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.369520    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 03:17:58.434398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.434364    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:58.434398 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.434390    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw9kk\" (UniqueName: \"kubernetes.io/projected/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f-kube-api-access-bw9kk\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:58.442183 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.442158    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:58.535412 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.535378    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\") pod \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") "
Nov 04 03:17:58.535654 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.535475    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\" (UniqueName: \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\") pod \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\" (UID: \"ac4af128-f2dd-4930-bb1f-baf8339f2bd4\") "
Nov 04 03:17:58.535863 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.535839    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume" pod "ac4af128-f2dd-4930-bb1f-baf8339f2bd4" (UID: "ac4af128-f2dd-4930-bb1f-baf8339f2bd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:17:58.537998 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.537963    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq" pod "ac4af128-f2dd-4930-bb1f-baf8339f2bd4" (UID: "ac4af128-f2dd-4930-bb1f-baf8339f2bd4"). InnerVolumeSpecName "kube-api-access-ntnmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:17:58.636770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.636743    6019 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-plugins-config-volume\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:58.636770 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.636768    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntnmq\" (UniqueName: \"kubernetes.io/projected/ac4af128-f2dd-4930-bb1f-baf8339f2bd4-kube-api-access-ntnmq\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:17:58.722432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.722389    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 03:17:58.728625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.728584    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8"]
Nov 04 03:17:58.980887 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:58.980804    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" path="/var/lib/kubelet/pods/40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f/volumes"
Nov 04 03:17:59.307325 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.307301    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"
Nov 04 03:17:59.307520 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.307300    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk" event={"ID":"ac4af128-f2dd-4930-bb1f-baf8339f2bd4","Type":"ContainerDied","Data":"c2791450cfaf25caa95d87f7ff4a43ba40b79a1f24fa976ff2950acace2df530"}
Nov 04 03:17:59.307520 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.307429    6019 scope.go:122] "RemoveContainer" containerID="1dcd8be61b57327bdf4ed70d6f2d4f21bad491e1caf80917d03b08fe51c00bc5"
Nov 04 03:17:59.470689 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.470652    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 03:17:59.520868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.520831    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk"]
Nov 04 03:17:59.522595 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.522568    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 03:17:59.522812 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.522782    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" podUID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" containerName="maas-api" containerID="cri-o://c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4" gracePeriod=30
Nov 04 03:17:59.607338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.607262    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:59.791746 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.791716    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 03:17:59.951649 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.951574    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") pod \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") "
Nov 04 03:17:59.951649 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.951608    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\" (UniqueName: \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\") pod \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") "
Nov 04 03:17:59.951649 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.951636    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\") pod \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") "
Nov 04 03:17:59.951870 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.951763    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\") pod \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\" (UID: \"fdb006a8-6c68-45e2-8eb6-ad8d5957032c\") "
Nov 04 03:17:59.952089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.952054    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca" pod "fdb006a8-6c68-45e2-8eb6-ad8d5957032c" (UID: "fdb006a8-6c68-45e2-8eb6-ad8d5957032c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:17:59.953938 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.953907    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs" pod "fdb006a8-6c68-45e2-8eb6-ad8d5957032c" (UID: "fdb006a8-6c68-45e2-8eb6-ad8d5957032c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:17:59.954064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.953994    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls" pod "fdb006a8-6c68-45e2-8eb6-ad8d5957032c" (UID: "fdb006a8-6c68-45e2-8eb6-ad8d5957032c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:17:59.954064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:17:59.954028    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7" pod "fdb006a8-6c68-45e2-8eb6-ad8d5957032c" (UID: "fdb006a8-6c68-45e2-8eb6-ad8d5957032c"). InnerVolumeSpecName "kube-api-access-n74s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:00.052869 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.052835    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:00.052869 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.052866    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:00.053121 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.052879    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n74s7\" (UniqueName: \"kubernetes.io/projected/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-kube-api-access-n74s7\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:00.053121 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.052893    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb006a8-6c68-45e2-8eb6-ad8d5957032c-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:00.314530 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.314495    6019 generic.go:356] "Generic (PLEG): container finished" podID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" containerID="c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4" exitCode=0
Nov 04 03:18:00.314718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.314541    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"
Nov 04 03:18:00.314718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.314588    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" event={"ID":"fdb006a8-6c68-45e2-8eb6-ad8d5957032c","Type":"ContainerDied","Data":"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"}
Nov 04 03:18:00.314718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.314643    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc" event={"ID":"fdb006a8-6c68-45e2-8eb6-ad8d5957032c","Type":"ContainerDied","Data":"09ccb7939ac49756ceb76664dc4d78ec6624de38a720375861a22fdbeafc25c0"}
Nov 04 03:18:00.314718 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.314669    6019 scope.go:122] "RemoveContainer" containerID="c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"
Nov 04 03:18:00.324331 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.324311    6019 scope.go:122] "RemoveContainer" containerID="c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"
Nov 04 03:18:00.324586 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:18:00.324568    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4\": container with ID starting with c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4 not found: ID does not exist" containerID="c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"
Nov 04 03:18:00.324634 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.324595    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4"} err="failed to get container status \"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4\": rpc error: code = NotFound desc = could not find container \"c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4\": container with ID starting with c3b72f4c18fa437686c5b6af51303c62783118fdb58fc47a95818300d105b6a4 not found: ID does not exist"
Nov 04 03:18:00.478381 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.478347    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 03:18:00.527783 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.527756    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-072ece-7474b6fdb-k5qfc"]
Nov 04 03:18:00.980485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.980457    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" path="/var/lib/kubelet/pods/ac4af128-f2dd-4930-bb1f-baf8339f2bd4/volumes"
Nov 04 03:18:00.980877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:00.980842    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" path="/var/lib/kubelet/pods/fdb006a8-6c68-45e2-8eb6-ad8d5957032c/volumes"
Nov 04 03:18:01.375434 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:01.375399    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:03.856893 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:03.856862    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:04.007834 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:04.007795    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:05.578324 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:05.578286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:06.078691 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:06.078642    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:06.386773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:06.386692    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:06.603616 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:18:06.603574    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:18:06.908387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:06.908355    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:09.593438 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:09.593359    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:09.801550 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:09.801459    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:09.947338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:09.947236    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:10.153409 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:10.151761    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:11.669275 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:11.669236    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:12.046422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:12.046387    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:12.550048 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:12.550020    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:13.121019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:13.120986    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:13.809093 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:13.809045    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:13.953036 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:13.952991    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:14.830226 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:14.830193    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:14.977886 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:14.977840    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:16.239167 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:16.239136    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:16.888055 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:16.888020    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:19.358262 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:19.358227    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:21.144513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:21.144423    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:22.986689 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:22.986650    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:25.078210 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:25.078172    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:25.251468 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:25.251437    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:26.682962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:26.682927    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:27.229328 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:27.229284    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:27.749425 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:27.749394    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:27.990715 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:27.990684    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:28.135779 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:28.135708    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:32.721111 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:32.721053    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 03:18:32.806957 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:32.806918    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:33.086565 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:33.086523    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:35.037657 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:35.037626    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:36.603111 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:18:36.603057    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:18:38.014179 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:38.014108    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:41.053010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:41.052976    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:41.252209 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:41.252178    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:42.419432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:42.419401    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:42.561958 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:42.561926    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:43.088979 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:43.088943    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:43.231759 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:43.231729    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:47.492347 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:47.492313    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:47.742581 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:47.742500    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main" containerID="cri-o://6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df" gracePeriod=45
Nov 04 03:18:47.984892 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:47.984869    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 03:18:48.092479 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092450    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092630 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092491    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092630 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092560    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092752 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092642    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\" (UniqueName: \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092752 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092689    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092752 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092732    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\") pod \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\" (UID: \"2f43d768-d8d6-4b1f-9b23-8e1d4652993a\") "
Nov 04 03:18:48.092918 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092786    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:48.092918 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092905    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:48.093027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.092962    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:48.093154 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.093126    6019 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tmp-dir\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.093154 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.093147    6019 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-home\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.093243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.093161    6019 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-model-cache\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.094979 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.094954    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:48.095150 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.095127    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:48.095223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.095204    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj" pod "2f43d768-d8d6-4b1f-9b23-8e1d4652993a" (UID: "2f43d768-d8d6-4b1f-9b23-8e1d4652993a"). InnerVolumeSpecName "kube-api-access-nt9nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:48.193672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.193641    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt9nj\" (UniqueName: \"kubernetes.io/projected/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-kube-api-access-nt9nj\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.193672 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.193670    6019 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-dshm\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.193861 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.193680    6019 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f43d768-d8d6-4b1f-9b23-8e1d4652993a-tls-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:18:48.552910 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.552876    6019 generic.go:356] "Generic (PLEG): container finished" podID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerID="6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df" exitCode=0
Nov 04 03:18:48.553342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.552925    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"
Nov 04 03:18:48.553342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.552962    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" event={"ID":"2f43d768-d8d6-4b1f-9b23-8e1d4652993a","Type":"ContainerDied","Data":"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"}
Nov 04 03:18:48.553342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.553005    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg" event={"ID":"2f43d768-d8d6-4b1f-9b23-8e1d4652993a","Type":"ContainerDied","Data":"439dad69968cf50895d4ec991a75113c0d871291c017f821e0645a6cace556e5"}
Nov 04 03:18:48.553342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.553020    6019 scope.go:122] "RemoveContainer" containerID="6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"
Nov 04 03:18:48.562629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.562613    6019 scope.go:122] "RemoveContainer" containerID="6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"
Nov 04 03:18:48.562895 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:18:48.562874    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df\": container with ID starting with 6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df not found: ID does not exist" containerID="6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"
Nov 04 03:18:48.562962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.562903    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df"} err="failed to get container status \"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df\": rpc error: code = NotFound desc = could not find container \"6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df\": container with ID starting with 6c6350b8b4f39516a4de3b98261e62085a57c00b4bb629750edaf4bec7c034df not found: ID does not exist"
Nov 04 03:18:48.715022 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.714987    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 03:18:48.764870 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.764841    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-203c35/premium-w4-203c35-kserve-dc5c4d6c4-vhvjg"]
Nov 04 03:18:48.980079 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:48.979989    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" path="/var/lib/kubelet/pods/2f43d768-d8d6-4b1f-9b23-8e1d4652993a/volumes"
Nov 04 03:18:52.162374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:52.162322    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:52.311488 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:52.311442    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:52.661333 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:52.661285    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:53.597282 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:53.597248    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:59.442577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:59.442541    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:59.585733 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:18:59.585694    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:00.742787 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:00.742740    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:06.024057 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:06.024026    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:19:06.036626 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:06.036609    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:19:06.602907 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:19:06.602869    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:19:06.845115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:06.845057    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:09.185089 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:09.185032    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:09.330743 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:09.330690    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:10.603649 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:10.603613    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:14.240426 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:14.240387    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:14.699570 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:14.699521    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:14.699917 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:14.699893    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 03:19:14.700164 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:14.700125    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy" containerID="cri-o://236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15" gracePeriod=30
Nov 04 03:19:16.838391 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:16.838363    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:16.987801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:16.987772    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:19.946851 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:19.946824    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 03:19:20.006316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006293    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006340    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006365    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006449 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006388    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006499    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006528    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006577    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006621    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006647    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\") pod \"e0472db5-6406-489a-bfdb-47586179bb33\" (UID: \"e0472db5-6406-489a-bfdb-47586179bb33\") "
Nov 04 03:19:20.006769 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006677    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:20.006928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006820    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:20.007063 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.006991    6019 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-data\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.007063 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.007014    6019 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.007063 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.007039    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:20.007285 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.007057    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:20.007285 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.007243    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:20.009258 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.009236    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:19:20.009258 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.009242    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:20.009432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.009401    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "kube-api-access-npzlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:20.009432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.009401    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token" pod "e0472db5-6406-489a-bfdb-47586179bb33" (UID: "e0472db5-6406-489a-bfdb-47586179bb33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:20.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107518    6019 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-istio-token\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107544    6019 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0472db5-6406-489a-bfdb-47586179bb33-istiod-ca-cert\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107554    6019 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-credential-socket\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107554 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107563    6019 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-workload-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107824 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107571    6019 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0472db5-6406-489a-bfdb-47586179bb33-istio-envoy\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107824 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107579    6019 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0472db5-6406-489a-bfdb-47586179bb33-istio-podinfo\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.107824 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.107589    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npzlb\" (UniqueName: \"kubernetes.io/projected/e0472db5-6406-489a-bfdb-47586179bb33-kube-api-access-npzlb\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:19:20.701941 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.701896    6019 generic.go:356] "Generic (PLEG): container finished" podID="e0472db5-6406-489a-bfdb-47586179bb33" containerID="236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15" exitCode=0
Nov 04 03:19:20.702177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.701969    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"
Nov 04 03:19:20.702177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.701978    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" event={"ID":"e0472db5-6406-489a-bfdb-47586179bb33","Type":"ContainerDied","Data":"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"}
Nov 04 03:19:20.702177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.702058    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm" event={"ID":"e0472db5-6406-489a-bfdb-47586179bb33","Type":"ContainerDied","Data":"caf31b11630d285dd81db27b5d5ec4ec0b5f92701029d455d620250bf3a2e0a9"}
Nov 04 03:19:20.702177 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.702101    6019 scope.go:122] "RemoveContainer" containerID="236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"
Nov 04 03:19:20.711681 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.711664    6019 scope.go:122] "RemoveContainer" containerID="236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"
Nov 04 03:19:20.711998 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:19:20.711980    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15\": container with ID starting with 236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15 not found: ID does not exist" containerID="236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"
Nov 04 03:19:20.712045 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.712007    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15"} err="failed to get container status \"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15\": rpc error: code = NotFound desc = could not find container \"236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15\": container with ID starting with 236fbaa99eafa738a7434c48d4e81085d7333ec804bfadcac4551f5ed6105a15 not found: ID does not exist"
Nov 04 03:19:20.866235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.866208    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 03:19:20.916102 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.916049    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm"]
Nov 04 03:19:20.980288 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:20.980205    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0472db5-6406-489a-bfdb-47586179bb33" path="/var/lib/kubelet/pods/e0472db5-6406-489a-bfdb-47586179bb33/volumes"
Nov 04 03:19:36.602504 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:19:36.602465    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:19:41.466338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:41.466256    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:43.560761 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:43.560729    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:47.046110 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046065    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"]
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046437    6019 cpu_manager.go:414] "containerMap: removing container" podUID="090014a4-3291-44d8-9784-e151bfd3cb0d" containerName="main"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046448    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="090014a4-3291-44d8-9784-e151bfd3cb0d" containerName="main"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046462    6019 cpu_manager.go:414] "containerMap: removing container" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046467    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046480    6019 cpu_manager.go:414] "containerMap: removing container" podUID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" containerName="maas-api"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046484    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" containerName="maas-api"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046493    6019 cpu_manager.go:414] "containerMap: removing container" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046498    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046505    6019 cpu_manager.go:414] "containerMap: removing container" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046509    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046517    6019 cpu_manager.go:414] "containerMap: removing container" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy"
Nov 04 03:19:47.046525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046521    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046536    6019 cpu_manager.go:414] "containerMap: removing container" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046541    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046551    6019 cpu_manager.go:414] "containerMap: removing container" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046556    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046562    6019 cpu_manager.go:414] "containerMap: removing container" podUID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046566    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046572    6019 cpu_manager.go:414] "containerMap: removing container" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046576    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="821911de-8e59-4e75-81ba-d1c3caba37a2" containerName="cleanup"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046585    6019 cpu_manager.go:414] "containerMap: removing container" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046590    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046595    6019 cpu_manager.go:414] "containerMap: removing container" podUID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" containerName="payload-pre-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046600    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" containerName="payload-pre-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046607    6019 cpu_manager.go:414] "containerMap: removing container" podUID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" containerName="payload-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046611    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" containerName="payload-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046618    6019 cpu_manager.go:414] "containerMap: removing container" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046622    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046629    6019 cpu_manager.go:414] "containerMap: removing container" podUID="03438791-5762-47bc-9304-03da49f041b4" containerName="test-curl-7027-cdfc76"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046634    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03438791-5762-47bc-9304-03da49f041b4" containerName="test-curl-7027-cdfc76"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046684    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="090014a4-3291-44d8-9784-e151bfd3cb0d" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046691    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0472db5-6406-489a-bfdb-47586179bb33" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046698    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdb006a8-6c68-45e2-8eb6-ad8d5957032c" containerName="maas-api"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046703    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="0405bd41-0c36-43a5-9aef-41f06a6bc22b" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046708    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="556dc3ec-4888-4b5d-909b-6a5023593b55" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046713    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc093e4d-e9ca-4bc2-8bf3-9c95dc748771" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046719    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="41a5b1fb-c31a-49e1-a4bb-9f7492a9e73e" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046724    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f43d768-d8d6-4b1f-9b23-8e1d4652993a" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046729    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="bea7cb70-ac11-4f0c-8139-cd4fa3f2bd89" containerName="main"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046734    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b2c92fc-84e0-4959-9eb2-59fcc6572539" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046738    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac4af128-f2dd-4930-bb1f-baf8339f2bd4" containerName="payload-pre-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046743    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="40f1e61b-dd6b-41b3-a84d-0aadb4a05c4f" containerName="payload-processing"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046748    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfda9917-4319-4230-89a0-7624e0236375" containerName="istio-proxy"
Nov 04 03:19:47.046899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.046758    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="03438791-5762-47bc-9304-03da49f041b4" containerName="test-curl-7027-cdfc76"
Nov 04 03:19:47.049711 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.049688    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052774 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052750    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052791    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052889 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052857    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052887    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052911    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.052970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.052931    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh6p5\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-kube-api-access-nh6p5\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.053099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.053034    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.053099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.053064    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.053196 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.053142    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.153935 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.153882    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.153935 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.153934    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154241 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.153976    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154241 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154135    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154241 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154201    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154254    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154302    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154334    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.154422 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154382    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.155220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154709    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.155220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154760    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh6p5\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-kube-api-access-nh6p5\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.155220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154789    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.155220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154798    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.155220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.154927    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.157307 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.157279    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.157413 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.157379    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.217877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.217842    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.218037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.218020    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh6p5\" (UniqueName: \"kubernetes.io/projected/16fcb15d-4bbc-4da9-9a13-e9d2a335a61d-kube-api-access-nh6p5\") pod \"maas-default-gateway-openshift-default-76597d6855-mtx78\" (UID: \"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.247644 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.247619    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"]
Nov 04 03:19:47.362623 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.362555    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:47.785705 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.785672    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"]
Nov 04 03:19:47.788859 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:19:47.788824    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fcb15d_4bbc_4da9_9a13_e9d2a335a61d.slice/crio-6930425ac3a9c56cc37bca34dd271db094cec3d31d737087221a4d2b4bcdb162 WatchSource:0}: Error finding container 6930425ac3a9c56cc37bca34dd271db094cec3d31d737087221a4d2b4bcdb162: Status 404 returned error can't find the container with id 6930425ac3a9c56cc37bca34dd271db094cec3d31d737087221a4d2b4bcdb162
Nov 04 03:19:47.791027 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.790997    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:19:47.791110 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.791062    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:19:47.791163 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.791111    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:19:47.823024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:47.823001    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" event={"ID":"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d","Type":"ContainerStarted","Data":"6930425ac3a9c56cc37bca34dd271db094cec3d31d737087221a4d2b4bcdb162"}
Nov 04 03:19:48.829023 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:48.828985    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" event={"ID":"16fcb15d-4bbc-4da9-9a13-e9d2a335a61d","Type":"ContainerStarted","Data":"84de470defb8e35d40b6e98aa99be34f7f37c23ae08a59817963786b31925b8d"}
Nov 04 03:19:48.999380 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:48.999317    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podStartSLOduration=2.9992985 podStartE2EDuration="2.9992985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:19:48.998182663 +0000 UTC m=+2744.139196070" watchObservedRunningTime="2026-11-04 03:19:48.9992985 +0000 UTC m=+2744.140311907"
Nov 04 03:19:49.373175 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:49.373133    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:49.373603 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:49.373577    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:49.373662 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:49.373635    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:50.367099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:50.366571    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:50.367099 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:50.366639    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:51.373483 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:51.373439    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:51.373919 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:51.373501    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:52.368574 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:52.368531    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:52.368877 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:52.368844    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:53.373101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:53.368506    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:53.373101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:53.368585    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:54.363386 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:54.363350    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:54.363593 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:54.363411    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:55.363419 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:55.363384    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 04 03:19:55.363826 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:55.363438    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 04 03:19:56.322224 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:56.322165    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:57.364382 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:57.364339    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:19:57.364807 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:57.364398    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:19:58.389805 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.389762    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:58.406277 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.406241    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:19:58.406448 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.406305    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:19:58.660933 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.660847    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:58.661281 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.661260    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:19:58.661947 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:19:58.661928    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78"
Nov 04 03:20:06.603045 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:20:06.603008    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:20:36.602652 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:20:36.602605    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:20:41.766327 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:20:41.766293    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:43.967879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:20:43.967765    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:44.667321 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:20:44.667276    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:20:44.667515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:20:44.667346    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:21:02.617598 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.617560    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"]
Nov 04 03:21:02.622360 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.622332    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.726349 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.726316    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"]
Nov 04 03:21:02.751013 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.750976    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751019    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751134    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751218 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751159    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751391 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751249    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4lqc\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-kube-api-access-g4lqc\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751391 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751311    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751406    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751432    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2030eefc-7cef-4652-97ae-f3fa1e560e13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.751471 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.751451    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852338 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852308    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852379    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852420    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2030eefc-7cef-4652-97ae-f3fa1e560e13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852548 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852443    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852716 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852588    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852716 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852640    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852716 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852699    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852734    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852758    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.852884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852775    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4lqc\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-kube-api-access-g4lqc\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.853047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.852956    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.853047 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.853002    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.853197 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.853107    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.853425 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.853198    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2030eefc-7cef-4652-97ae-f3fa1e560e13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.854924 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.854902    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.855221 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.855204    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.913699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.913644    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4lqc\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-kube-api-access-g4lqc\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.913810 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.913740    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2030eefc-7cef-4652-97ae-f3fa1e560e13-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-hk6mk\" (UID: \"2030eefc-7cef-4652-97ae-f3fa1e560e13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:02.937063 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:02.937043    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:03.365132 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:03.365099    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"]
Nov 04 03:21:03.368307 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:21:03.368274    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2030eefc_7cef_4652_97ae_f3fa1e560e13.slice/crio-0a0571dee594f0066f5d461b0ce36e85adfd32c72eeacf9e14a242aca50911e9 WatchSource:0}: Error finding container 0a0571dee594f0066f5d461b0ce36e85adfd32c72eeacf9e14a242aca50911e9: Status 404 returned error can't find the container with id 0a0571dee594f0066f5d461b0ce36e85adfd32c72eeacf9e14a242aca50911e9
Nov 04 03:21:03.370318 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:03.370288    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:21:03.370390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:03.370354    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:21:03.370390 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:03.370384    6019 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 04 03:21:04.186509 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.186474    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" event={"ID":"2030eefc-7cef-4652-97ae-f3fa1e560e13","Type":"ContainerStarted","Data":"2acdb8f1d35860fe158be30c1f18237cb0e44f2b7fccbd3c1757f663b62a677d"}
Nov 04 03:21:04.186915 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.186524    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" event={"ID":"2030eefc-7cef-4652-97ae-f3fa1e560e13","Type":"ContainerStarted","Data":"0a0571dee594f0066f5d461b0ce36e85adfd32c72eeacf9e14a242aca50911e9"}
Nov 04 03:21:04.308055 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.307998    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podStartSLOduration=2.30798216 podStartE2EDuration="2.30798216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:21:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:21:04.306335826 +0000 UTC m=+2819.447349257" watchObservedRunningTime="2026-11-04 03:21:04.30798216 +0000 UTC m=+2819.448995566"
Nov 04 03:21:04.938302 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.938267    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:04.939762 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.939739    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:04.939885 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:04.939780    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:05.938500 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:05.938459    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:05.938924 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:05.938561    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:06.605474 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:21:06.605428    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:21:06.938533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:06.938449    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:06.938931 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:06.938529    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:07.938205 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:07.938162    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:07.938399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:07.938227    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:08.937518 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:08.937477    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:08.938024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:08.937538    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:09.938536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:09.938453    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:09.938536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:09.938511    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:10.937884 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:10.937843    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 04 03:21:10.938088 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:10.937932    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 04 03:21:12.978286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:12.978247    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:21:12.978899 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:12.978311    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:21:14.019237 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:14.019202    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:21:14.019629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:14.019253    6019 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:21:14.188531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:14.188498    6019 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:14.188921 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:14.188899    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:14.189629 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:14.189615    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk"
Nov 04 03:21:36.603035 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:21:36.602994    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:21:42.128962 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:42.128925    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:43.264317 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:43.264286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:44.336184 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:44.336142    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:48.494655 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:48.494621    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:48.636771 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:21:48.636737    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:06.603755 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:22:06.603709    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:22:36.604429 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:22:36.604384    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:22:42.481376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:22:42.481283    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:44.681885 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:22:44.681838    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:58.710721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:22:58.710682    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:06.640676 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:23:06.640642    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:23:30.237240 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:30.237190    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:23:30.237747 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:30.237260    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:23:36.602376 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:23:36.602328    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:23:42.833763 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:42.833706    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:45.036906 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:45.036845    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:47.846238 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:47.846193    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:23:47.846636 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:47.846259    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:23:57.462153 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:23:57.462113    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:06.060020 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:24:06.059995    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:24:06.071597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:24:06.071574    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:24:06.602883 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:24:06.602845    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:24:36.602663 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:24:36.602614    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:24:43.239312 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:24:43.239279    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:43.383836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:24:43.383803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:45.392333 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:24:45.392278    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:06.602983 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:25:06.602939    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:25:36.601718 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:25:36.601681    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:25:44.532640 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:25:44.532537    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:45.745966 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:25:45.745934    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:55.273842 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:25:55.273788    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:55.882096 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:25:55.875803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:56.236676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:25:56.236590    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:00.324608 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:00.324560    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:00.513245 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:00.513213    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:00.658567 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:00.658491    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:03.628013 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:03.627979    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:03.806513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:03.806478    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:06.602096 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:26:06.602038    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:26:06.846533 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:06.846496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:07.009513 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:07.009481    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:07.157667 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:07.157634    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:11.878487 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:11.878449    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.797245 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:16.797208    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:21.682371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:21.682330    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:25.091457 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:25.091424    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:25.687722 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:25.687688    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:25.835424 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:25.835390    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:26.888948 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:26.888916    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:28.469996 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:28.469941    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:31.786674 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:31.786641    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:36.602715 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:26:36.602672    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:26:36.684026 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:36.683906    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:41.580313 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:41.580279    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:43.887329 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:43.887292    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:46.080943 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:46.080881    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:46.491664 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:46.491584    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:51.374705 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:51.374670    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:56.288452 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:26:56.288410    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:01.177006 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:01.176970    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:06.081140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:06.081049    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:06.602542 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:27:06.602504    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:27:10.979693 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:10.979609    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:15.890010 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:15.889971    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:20.707291 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:20.707251    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:24.712044 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:24.712006    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:25.170739 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:25.170703    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:25.578676 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:25.578639    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:29.778772 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:29.778739    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:30.471572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:30.471535    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:31.227019 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:31.226984    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:32.111115 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:32.111066    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:32.257303 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:32.257268    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:34.945736 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:34.945701    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:36.604895 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:27:36.604856    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:27:37.028406 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:37.028373    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:37.172061 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:37.172026    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:44.239371 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:44.239333    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:46.451409 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:27:46.451375    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:06.151024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:06.150983    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:06.602787 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:28:06.602753    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:28:06.716467 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:06.716427    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:06.864578 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:06.864494    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:07.334441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:07.334400    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:08.588009 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:08.587969    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:09.018748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:09.018715    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:09.681983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:09.681945    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:13.355720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:13.355682    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:13.498731 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:13.498702    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:30.037342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:30.037310    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:36.603106 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:28:36.603040    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:28:44.600016 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:44.599932    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:46.797994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:28:46.797964    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:06.091368 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:06.091336    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:29:06.104880 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:06.104856    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:29:06.601823 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:29:06.601782    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:29:36.602340 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:29:36.602299    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:29:45.003744 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:45.003714    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:47.156836 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:47.156797    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:52.641532 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:52.641495    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:54.496342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:54.496301    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:54.640334 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:29:54.640300    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:06.602193 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:30:06.602155    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:30:36.206748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:36.206711    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:36.611611 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:30:36.611570    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:30:36.807365 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:36.807323    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:36.980839 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:36.980678    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:39.161170 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:39.161136    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:41.038182 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:41.038137    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:41.206035 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:41.206003    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:41.367363 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:41.367286    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:44.312435 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:44.312381    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:44.487213 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:44.487168    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:45.300597 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:45.300556    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:47.426284 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:47.426243    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:47.566855 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:47.566819    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:47.714138 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:47.714026    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:52.477158 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:52.477096    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:57.404087 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:30:57.404043    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:02.285912 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:02.285869    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:05.836625 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:05.836593    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:06.445184 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:06.445051    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:06.589507 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:06.589471    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:06.602938 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:31:06.602900    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:31:06.734404 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:06.734324    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:07.401021 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:07.400988    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:09.890186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:09.890148    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:12.283065 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:12.283031    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:17.200341 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:17.200301    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:22.140314 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:22.139505    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:26.990949 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:26.990916    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:31.875465 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:31.875417    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:35.883345 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:35.883304    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:36.602966 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:31:36.602930    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:31:36.791301 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:36.791272    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:41.673157 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:41.673019    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:45.657059 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:45.657014    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:46.646749 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:46.646710    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:47.619145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:47.619098    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:47.867265 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:47.867234    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:51.376742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:51.376684    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:56.291978 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:31:56.291940    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:01.175785 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:01.175743    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:05.731031 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:05.730978    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:06.085619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:06.085551    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:06.228028 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:06.227946    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:06.603031 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:32:06.602995    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:32:10.784900 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:10.784862    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:10.982383 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:10.982331    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:12.330206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:12.330170    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:12.473530 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:12.473496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:12.681107 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:12.680994    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:15.533903 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:15.533867    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:18.486316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:18.486239    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:18.630748 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:18.630698    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:36.602735 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:32:36.602696    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:32:46.009577 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:46.009531    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:47.384659 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:47.384626    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:47.537544 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:47.537496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:48.094662 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:48.094625    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:48.245432 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:48.245396    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:49.714700 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:49.714669    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:49.861235 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:49.861187    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:50.004774 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:50.004747    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:53.731841 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:53.731803    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:53.874684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:53.874641    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:54.842702 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:54.842666    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:59.691373 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:32:59.691338    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:04.493323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:04.493270    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:06.603679 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:33:06.603636    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:33:09.398186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:09.398148    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:14.194198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:14.194152    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:18.465571 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:18.465537    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:19.992342 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:19.992313    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:20.271936 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:20.271903    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:20.923125 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:20.923065    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:28.170613 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:28.170575    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:36.602870 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:33:36.602833    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:33:46.363651 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:46.363615    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:48.578594 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:48.578560    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:55.076508 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:33:55.076471    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:02.845243 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:02.845202    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-74hkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:34:02.845751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:02.845267    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj" podUID="4afac37a-d227-4dd0-8f7d-b7c276c4974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:34:06.131720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:06.131688    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:34:06.138383 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:06.138361    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:34:06.603732 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:34:06.603691    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:34:36.603455 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:34:36.603412    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:34:42.402925 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:42.402847    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:46.763140 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:46.763102    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:49.093852 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:49.093804    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:49.243330 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:34:49.243288    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:06.602921 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:35:06.602882    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:35:14.715256 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:14.715201    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mtx78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:35:14.716347 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:14.716314    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78" podUID="16fcb15d-4bbc-4da9-9a13-e9d2a335a61d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:35:36.603356 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:35:36.603316    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:35:37.507720 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:37.507643    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:38.117848 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:38.117817    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:38.350697 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:38.350661    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:39.004223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.004193    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"]
Nov 04 03:35:39.007721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.007704    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.110754 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.110723    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"]
Nov 04 03:35:39.142587 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.142550    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-maas-api-tls\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.142999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.142606    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a253f521-9fd1-4f9e-9aff-b063aa7362f7-openshift-service-ca\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.142999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.142669    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-metrics-certs\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.142999 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.142688    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpbbn\" (UniqueName: \"kubernetes.io/projected/a253f521-9fd1-4f9e-9aff-b063aa7362f7-kube-api-access-jpbbn\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.243684 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.243640    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-maas-api-tls\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.243901 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.243726    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a253f521-9fd1-4f9e-9aff-b063aa7362f7-openshift-service-ca\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.243901 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.243750    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-metrics-certs\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.243901 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.243769    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpbbn\" (UniqueName: \"kubernetes.io/projected/a253f521-9fd1-4f9e-9aff-b063aa7362f7-kube-api-access-jpbbn\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.244527 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.244466    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a253f521-9fd1-4f9e-9aff-b063aa7362f7-openshift-service-ca\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.246407 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.246383    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-metrics-certs\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.246544 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.246522    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a253f521-9fd1-4f9e-9aff-b063aa7362f7-maas-api-tls\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.305220 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.305158    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpbbn\" (UniqueName: \"kubernetes.io/projected/a253f521-9fd1-4f9e-9aff-b063aa7362f7-kube-api-access-jpbbn\") pod \"maas-api-78d5c4d8b5-8pq24\" (UID: \"a253f521-9fd1-4f9e-9aff-b063aa7362f7\") " pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.320850 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.320827    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:39.694399 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.694374    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"]
Nov 04 03:35:39.696633 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:35:39.696607    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda253f521_9fd1_4f9e_9aff_b063aa7362f7.slice/crio-9c2cf217440cd30f1f07489d4a32fcdf11bb414d22566b0a6a4f1c1f3c95d57c WatchSource:0}: Error finding container 9c2cf217440cd30f1f07489d4a32fcdf11bb414d22566b0a6a4f1c1f3c95d57c: Status 404 returned error can't find the container with id 9c2cf217440cd30f1f07489d4a32fcdf11bb414d22566b0a6a4f1c1f3c95d57c
Nov 04 03:35:39.697847 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:39.697828    6019 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:35:40.698751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:40.698716    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24" event={"ID":"a253f521-9fd1-4f9e-9aff-b063aa7362f7","Type":"ContainerStarted","Data":"b0b096ed0723e4bf50505aae0ccf6d3aea21cffa32d0a494dd9fb8516884d7a7"}
Nov 04 03:35:40.699259 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:40.698765    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24" event={"ID":"a253f521-9fd1-4f9e-9aff-b063aa7362f7","Type":"ContainerStarted","Data":"9c2cf217440cd30f1f07489d4a32fcdf11bb414d22566b0a6a4f1c1f3c95d57c"}
Nov 04 03:35:40.699259 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:40.698804    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:40.846519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:40.846421    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24" podStartSLOduration=1.979695531 podStartE2EDuration="2.846406052s" totalImagesPullingTime="866.710521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:40.845609674 +0000 UTC m=+3695.986623082" watchObservedRunningTime="2026-11-04 03:35:40.846406052 +0000 UTC m=+3695.987419458"
Nov 04 03:35:42.116645 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:42.116602    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:42.289038 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:42.288993    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:42.442932 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:42.442854    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:45.423046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:45.423013    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:45.610805 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:45.610771    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:46.369037 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.369000    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:46.540536 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.540508    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:46.686149 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.686052    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:46.710911 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.710885    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78d5c4d8b5-8pq24"
Nov 04 03:35:46.862307 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.862268    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 03:35:46.862507 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:46.862466    6019 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" podUID="5971c81b-e937-4ae2-b565-9348e7762e36" containerName="maas-api" containerID="cri-o://106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d" gracePeriod=30
Nov 04 03:35:47.066085 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:47.066042    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:48.130466 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.130440    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 03:35:48.231466 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.231371    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\") pod \"5971c81b-e937-4ae2-b565-9348e7762e36\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") "
Nov 04 03:35:48.231466 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.231424    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\") pod \"5971c81b-e937-4ae2-b565-9348e7762e36\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") "
Nov 04 03:35:48.231695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.231498    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\" (UniqueName: \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\") pod \"5971c81b-e937-4ae2-b565-9348e7762e36\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") "
Nov 04 03:35:48.231695 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.231532    6019 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\") pod \"5971c81b-e937-4ae2-b565-9348e7762e36\" (UID: \"5971c81b-e937-4ae2-b565-9348e7762e36\") "
Nov 04 03:35:48.231891 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.231863    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca" pod "5971c81b-e937-4ae2-b565-9348e7762e36" (UID: "5971c81b-e937-4ae2-b565-9348e7762e36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:35:48.233857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.233829    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv" pod "5971c81b-e937-4ae2-b565-9348e7762e36" (UID: "5971c81b-e937-4ae2-b565-9348e7762e36"). InnerVolumeSpecName "kube-api-access-99trv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:35:48.233857 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.233851    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs" pod "5971c81b-e937-4ae2-b565-9348e7762e36" (UID: "5971c81b-e937-4ae2-b565-9348e7762e36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:35:48.234030 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.233830    6019 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls" pod "5971c81b-e937-4ae2-b565-9348e7762e36" (UID: "5971c81b-e937-4ae2-b565-9348e7762e36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:35:48.333024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.332988    6019 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-maas-api-tls\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:35:48.333024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.333019    6019 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5971c81b-e937-4ae2-b565-9348e7762e36-openshift-service-ca\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:35:48.333024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.333030    6019 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99trv\" (UniqueName: \"kubernetes.io/projected/5971c81b-e937-4ae2-b565-9348e7762e36-kube-api-access-99trv\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:35:48.333280 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.333039    6019 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5971c81b-e937-4ae2-b565-9348e7762e36-metrics-certs\") on node \"ip-10-0-128-201.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 03:35:48.741002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.740969    6019 generic.go:356] "Generic (PLEG): container finished" podID="5971c81b-e937-4ae2-b565-9348e7762e36" containerID="106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d" exitCode=0
Nov 04 03:35:48.741002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.741005    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" event={"ID":"5971c81b-e937-4ae2-b565-9348e7762e36","Type":"ContainerDied","Data":"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"}
Nov 04 03:35:48.741316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.741031    6019 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"
Nov 04 03:35:48.741316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.741050    6019 scope.go:122] "RemoveContainer" containerID="106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"
Nov 04 03:35:48.741316 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.741041    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn" event={"ID":"5971c81b-e937-4ae2-b565-9348e7762e36","Type":"ContainerDied","Data":"8c9c3437517e34f25eb2b00827591f906c186a36470c6e2dd9dfb9e202ce82a1"}
Nov 04 03:35:48.750813 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.750795    6019 scope.go:122] "RemoveContainer" containerID="106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"
Nov 04 03:35:48.751095 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:35:48.751061    6019 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d\": container with ID starting with 106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d not found: ID does not exist" containerID="106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"
Nov 04 03:35:48.751173 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.751108    6019 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d"} err="failed to get container status \"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d\": rpc error: code = NotFound desc = could not find container \"106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d\": container with ID starting with 106cc5f23a2fa978a110dbbbee52d418530eb1dac251a9c460d65f39a191ed6d not found: ID does not exist"
Nov 04 03:35:48.905781 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.905744    6019 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 03:35:48.955163 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.955133    6019 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b4f86cc7-5rqnn"]
Nov 04 03:35:48.980778 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:48.980752    6019 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5971c81b-e937-4ae2-b565-9348e7762e36" path="/var/lib/kubelet/pods/5971c81b-e937-4ae2-b565-9348e7762e36/volumes"
Nov 04 03:35:49.443654 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:49.443621    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:51.782710 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:51.782639    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:56.723175 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:56.723130    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:57.782704 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:57.782660    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:57.929773 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:35:57.929735    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:01.605729 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:01.605683    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.602220 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:36:06.602177    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:36:06.871784 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:06.871687    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:12.318129 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:12.318065    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:17.384181 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:17.384116    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:22.282721 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:22.282685    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:25.361028 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:25.360984    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:25.508196 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:25.508162    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:27.188155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:27.188125    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:30.488515 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:30.488480    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:30.665525 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:30.665492    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:32.120951 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:32.120916    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:36.402572 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:36.402532    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:36.602114 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:36:36.602050    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:36:37.945082 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:37.945016    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:38.228416 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:38.228328    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:39.628507 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:39.628447    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:40.873916 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:40.873868    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:41.460933 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:41.460895    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:41.746326 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:41.746227    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:46.093085 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:46.093025    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:46.266286 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:46.266249    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:46.411425 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:46.411325    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:46.590520 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:46.590480    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:47.868294 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:47.868261    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:49.369192 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:49.369157    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:49.543932 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:49.543901    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:49.867210 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:49.867143    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:53.255609 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:53.255539    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:53.421276 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:53.421234    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:53.567450 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:53.567354    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:58.375485 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:36:58.375425    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:03.297020 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:03.296989    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:06.602170 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:37:06.602134    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:37:08.181604 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:08.181564    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:11.003046 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:11.003011    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:13.406250 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:13.406206    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:18.275551 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:18.275496    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:23.198145 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:23.198103    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:28.078941 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:28.078902    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:32.988475 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:32.988416    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:36.601861 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:37:36.601827    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:37:37.887653 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:37.887585    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:42.882025 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:42.881909    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:47.690704 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:47.690650    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:47.832719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:47.832682    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:49.603155 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:49.603093    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:49.745387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:49.745349    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:50.163619 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:50.163589    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:52.583369 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:52.583315    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:57.396739 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:37:57.396678    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:02.291494 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:02.291420    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:05.689825 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:05.689783    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:06.603908 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:38:06.603866    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:38:07.183775 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:07.183739    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:12.096024 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:12.095973    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:16.986340 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:16.986303    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:21.931698 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:21.931619    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:26.779970 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:26.779887    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:31.622004 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:31.621969    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:31.766988 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:31.766954    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:36.580926 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:36.580873    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:36.602934 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:38:36.602887    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:38:37.763519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:37.763476    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:38.668198 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:38.668162    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:38.811295 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:38.811253    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:41.198655 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:41.198597    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:43.435441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:43.435402    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:43.584450 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:43.584399    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:45.045661 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:45.045593    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:45.684519 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:45.684472    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:48.129124 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:48.129062    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:48.276895 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:48.276857    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:50.519101 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:50.519036    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:54.874426 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:54.874381    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:55.016202 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:38:55.016170    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:06.239660 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:06.239627    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:39:06.244463 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:06.244440    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7lsk_d3cd0132-b993-48e2-8d27-e4a0145bf29d/kube-multus/0.log"
Nov 04 03:39:06.640946 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:39:06.640912    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:39:30.233302 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:30.233259    6019 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-hk6mk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:39:30.233715 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:30.233331    6019 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk" podUID="2030eefc-7cef-4652-97ae-f3fa1e560e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:36.602725 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:39:36.602687    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:39:44.746330 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:44.746290    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:48.542003 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:48.541968    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:50.871853 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:39:50.871819    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:06.602891 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:40:06.602850    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:40:08.390223 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:08.390192    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gpx6l_369c87dc-c619-43ea-a902-b89e7bf348a5/manager/0.log"
Nov 04 03:40:08.950982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:08.950949    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gbv2g_a72b7dac-d9ce-4a3b-9557-9db5bab724f6/manager/0.log"
Nov 04 03:40:09.762351 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:09.762323    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-qvtns_fdc44dab-8025-4e79-b9a6-fc9f94f70cd0/server/0.log"
Nov 04 03:40:11.854868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:11.854831    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-78d5c4d8b5-8pq24_a253f521-9fd1-4f9e-9aff-b063aa7362f7/maas-api/0.log"
Nov 04 03:40:13.040493 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:13.040464    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-7k5jm_fadfa4ac-e093-4930-9750-0bca1446d2bf/postgres/0.log"
Nov 04 03:40:16.050774 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:16.050746    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-48kqg_c325cb7f-d5e1-4efd-84db-0dfec36c5878/kuadrant-console-plugin/0.log"
Nov 04 03:40:18.665374 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:18.665341    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zch9j_b668da5e-cfe9-47fa-9f63-16cdea56c0bc/discovery/0.log"
Nov 04 03:40:19.032565 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:19.032533    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54497bf445-q5p8t_189d27e4-0565-4c01-836a-ed99a907f34c/kube-auth-proxy/0.log"
Nov 04 03:40:19.484441 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:19.484360    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-74hkj_4afac37a-d227-4dd0-8f7d-b7c276c4974c/istio-proxy/0.log"
Nov 04 03:40:20.473002 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:20.472971    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-hk6mk_2030eefc-7cef-4652-97ae-f3fa1e560e13/istio-proxy/0.log"
Nov 04 03:40:21.239288 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:21.239209    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-mtx78_16fcb15d-4bbc-4da9-9a13-e9d2a335a61d/istio-proxy/0.log"
Nov 04 03:40:36.602299 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:40:36.602262    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:40:36.858654 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:36.858572    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gpx6l_369c87dc-c619-43ea-a902-b89e7bf348a5/manager/0.log"
Nov 04 03:40:37.415014 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:37.414984    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gbv2g_a72b7dac-d9ce-4a3b-9557-9db5bab724f6/manager/0.log"
Nov 04 03:40:38.145357 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:38.145329    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-qvtns_fdc44dab-8025-4e79-b9a6-fc9f94f70cd0/server/0.log"
Nov 04 03:40:39.601751 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:39.601677    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-78d5c4d8b5-8pq24_a253f521-9fd1-4f9e-9aff-b063aa7362f7/maas-api/0.log"
Nov 04 03:40:40.526627 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:40.526597    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-7k5jm_fadfa4ac-e093-4930-9750-0bca1446d2bf/postgres/0.log"
Nov 04 03:40:42.738451 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:42.738418    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-48kqg_c325cb7f-d5e1-4efd-84db-0dfec36c5878/kuadrant-console-plugin/0.log"
Nov 04 03:40:44.650868 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:44.650837    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zch9j_b668da5e-cfe9-47fa-9f63-16cdea56c0bc/discovery/0.log"
Nov 04 03:40:44.929319 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:44.929244    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54497bf445-q5p8t_189d27e4-0565-4c01-836a-ed99a907f34c/kube-auth-proxy/0.log"
Nov 04 03:40:45.402095 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:45.402049    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-74hkj_4afac37a-d227-4dd0-8f7d-b7c276c4974c/istio-proxy/0.log"
Nov 04 03:40:46.371647 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:46.371616    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-hk6mk_2030eefc-7cef-4652-97ae-f3fa1e560e13/istio-proxy/0.log"
Nov 04 03:40:47.254084 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:47.254050    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-mtx78_16fcb15d-4bbc-4da9-9a13-e9d2a335a61d/istio-proxy/0.log"
Nov 04 03:40:48.844206 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:48.844173    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:51.226928 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:40:51.226893    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:06.604298 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:41:06.604261    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:41:25.157283 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:25.157246    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:36.602148 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:41:36.602105    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:41:49.202719 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:49.202680    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:49.978537 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:49.978500    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:51.582340 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:51.582309    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:56.573967 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:56.573932    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:56.722665 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:41:56.722634    6019 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:06.194620 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:06.194583    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8kq82_43a315e6-74d9-46f6-afad-3f9f3d4f98e0/global-pull-secret-syncer/0.log"
Nov 04 03:42:06.602732 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:42:06.602696    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:42:08.745578 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:08.745543    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l8xbh_def07bfa-aa26-4365-b160-67835f20cbd5/konnectivity-agent/0.log"
Nov 04 03:42:10.108827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:10.108744    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-201.us-west-1.compute.internal_b41ac5ecba6dc9accfa45177979cc284/haproxy/0.log"
Nov 04 03:42:16.723367 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:16.723339    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-48kqg_c325cb7f-d5e1-4efd-84db-0dfec36c5878/kuadrant-console-plugin/0.log"
Nov 04 03:42:17.058578 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.058549    6019 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"]
Nov 04 03:42:17.058944 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.058932    6019 cpu_manager.go:414] "containerMap: removing container" podUID="5971c81b-e937-4ae2-b565-9348e7762e36" containerName="maas-api"
Nov 04 03:42:17.058996 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.058945    6019 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5971c81b-e937-4ae2-b565-9348e7762e36" containerName="maas-api"
Nov 04 03:42:17.059039 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.059013    6019 memory_manager.go:365] "RemoveStaleState removing state" podUID="5971c81b-e937-4ae2-b565-9348e7762e36" containerName="maas-api"
Nov 04 03:42:17.061995 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.061979    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.121805 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.121774    6019 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wjk9m\"/\"default-dockercfg-fdwhv\""
Nov 04 03:42:17.171323 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.171293    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjk9m\"/\"kube-root-ca.crt\""
Nov 04 03:42:17.171484 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.171332    6019 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjk9m\"/\"openshift-service-ca.crt\""
Nov 04 03:42:17.172715 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.172694    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"]
Nov 04 03:42:17.218879 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.218830    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-lib-modules\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.219064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.218937    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-podres\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.219064 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.218966    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-proc\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.219213 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.219091    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qs58\" (UniqueName: \"kubernetes.io/projected/18e73c6d-b5f7-4d23-9491-1c27cca75732-kube-api-access-8qs58\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.219213 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.219114    6019 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-sys\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320604    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-lib-modules\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320653    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-podres\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320694 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320670    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-proc\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320722    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qs58\" (UniqueName: \"kubernetes.io/projected/18e73c6d-b5f7-4d23-9491-1c27cca75732-kube-api-access-8qs58\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320741    6019 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-sys\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320758    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-podres\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320757    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-lib-modules\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320810    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-proc\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.320946 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.320836    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18e73c6d-b5f7-4d23-9491-1c27cca75732-sys\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.383249 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.383216    6019 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qs58\" (UniqueName: \"kubernetes.io/projected/18e73c6d-b5f7-4d23-9491-1c27cca75732-kube-api-access-8qs58\") pod \"perf-node-gather-daemonset-nd8pf\" (UID: \"18e73c6d-b5f7-4d23-9491-1c27cca75732\") " pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:17.674425 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:17.674346    6019 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:18.046258 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:18.046219    6019 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"]
Nov 04 03:42:18.049804 ip-10-0-128-201 kubenswrapper[6019]: W1104 03:42:18.049778    6019 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18e73c6d_b5f7_4d23_9491_1c27cca75732.slice/crio-9e0f08865b82ad27cfb595c1998885f2ea12a505a7deb669daa426fdd27aca7d WatchSource:0}: Error finding container 9e0f08865b82ad27cfb595c1998885f2ea12a505a7deb669daa426fdd27aca7d: Status 404 returned error can't find the container with id 9e0f08865b82ad27cfb595c1998885f2ea12a505a7deb669daa426fdd27aca7d
Nov 04 03:42:18.594055 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:18.594018    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf" event={"ID":"18e73c6d-b5f7-4d23-9491-1c27cca75732","Type":"ContainerStarted","Data":"8a135bfb44fa94eabb973ffc66b47334bebf8792e571c40e3e0bf22a7d568a11"}
Nov 04 03:42:18.594273 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:18.594128    6019 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf" event={"ID":"18e73c6d-b5f7-4d23-9491-1c27cca75732","Type":"ContainerStarted","Data":"9e0f08865b82ad27cfb595c1998885f2ea12a505a7deb669daa426fdd27aca7d"}
Nov 04 03:42:18.594273 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:18.594245    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:18.712755 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:18.712702    6019 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf" podStartSLOduration=1.712687835 podStartE2EDuration="1.712687835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:42:18.711622535 +0000 UTC m=+4093.852635979" watchObservedRunningTime="2026-11-04 03:42:18.712687835 +0000 UTC m=+4093.853701241"
Nov 04 03:42:20.917268 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:20.917240    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/alertmanager/0.log"
Nov 04 03:42:21.347686 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:21.347659    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/config-reloader/0.log"
Nov 04 03:42:21.769801 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:21.769774    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/kube-rbac-proxy-web/0.log"
Nov 04 03:42:22.195785 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:22.195707    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/kube-rbac-proxy/0.log"
Nov 04 03:42:22.620904 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:22.620879    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/kube-rbac-proxy-metric/0.log"
Nov 04 03:42:23.041534 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:23.041509    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/prom-label-proxy/0.log"
Nov 04 03:42:23.462890 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:23.462815    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7faba029-68fd-4da7-a4da-5cb22dd441bb/init-config-reloader/0.log"
Nov 04 03:42:24.318430 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:24.318395    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-95sfm_f7913919-20f9-4b2c-b1fe-81c2552e5c77/kube-state-metrics/0.log"
Nov 04 03:42:24.609362 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:24.609288    6019 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wjk9m/perf-node-gather-daemonset-nd8pf"
Nov 04 03:42:24.750699 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:24.750673    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-95sfm_f7913919-20f9-4b2c-b1fe-81c2552e5c77/kube-rbac-proxy-main/0.log"
Nov 04 03:42:25.172199 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:25.172170    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-95sfm_f7913919-20f9-4b2c-b1fe-81c2552e5c77/kube-rbac-proxy-self/0.log"
Nov 04 03:42:25.596738 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:25.596711    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bdc474876-v9z8c_ad378d3a-66cb-406a-aa92-ff17ad170512/metrics-server/0.log"
Nov 04 03:42:26.023994 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:26.023964    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-4jh5c_7767fc65-d7d9-42ce-8787-7b7c9af5ee4f/monitoring-plugin/0.log"
Nov 04 03:42:27.730376 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:27.730345    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cggz4_7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/node-exporter/0.log"
Nov 04 03:42:28.152914 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:28.152890    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cggz4_7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/kube-rbac-proxy/0.log"
Nov 04 03:42:28.577040 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:28.577014    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cggz4_7ce3cee4-69e7-4743-ae18-ee9921c5c7d9/init-textfile/0.log"
Nov 04 03:42:30.272531 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:30.272504    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zvwwf_b675972a-9d65-4a98-9fbe-79b925302105/kube-rbac-proxy-main/0.log"
Nov 04 03:42:30.698386 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:30.698314    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zvwwf_b675972a-9d65-4a98-9fbe-79b925302105/kube-rbac-proxy-self/0.log"
Nov 04 03:42:31.120983 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:31.120949    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zvwwf_b675972a-9d65-4a98-9fbe-79b925302105/openshift-state-metrics/0.log"
Nov 04 03:42:34.518229 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:34.518200    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-czjvg_3223c5ed-97b7-46fe-baba-9e54b1b819bd/prometheus-operator/0.log"
Nov 04 03:42:34.936514 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:34.936431    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-czjvg_3223c5ed-97b7-46fe-baba-9e54b1b819bd/kube-rbac-proxy/0.log"
Nov 04 03:42:35.797761 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:35.797732    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8658654fb-59dbg_eb2814b3-9ba3-4b02-ad23-776c9f286ef3/telemeter-client/0.log"
Nov 04 03:42:36.217882 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:36.217806    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8658654fb-59dbg_eb2814b3-9ba3-4b02-ad23-776c9f286ef3/reload/0.log"
Nov 04 03:42:36.602516 ip-10-0-128-201 kubenswrapper[6019]: E1104 03:42:36.602480    6019 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r8bmk" containerName="extractor"
Nov 04 03:42:36.637827 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:36.637803    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8658654fb-59dbg_eb2814b3-9ba3-4b02-ad23-776c9f286ef3/kube-rbac-proxy/0.log"
Nov 04 03:42:37.064403 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:37.064373    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/thanos-query/0.log"
Nov 04 03:42:37.743742 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:37.743714    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/kube-rbac-proxy-web/0.log"
Nov 04 03:42:38.165306 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:38.165281    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/kube-rbac-proxy/0.log"
Nov 04 03:42:38.594186 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:38.594159    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/prom-label-proxy/0.log"
Nov 04 03:42:39.016387 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:39.016354    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/kube-rbac-proxy-rules/0.log"
Nov 04 03:42:39.467274 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:39.467205    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686fd66bbf-qv4rg_ab6381ce-e265-4942-9c1b-0652a4cd88c3/kube-rbac-proxy-metrics/0.log"
Nov 04 03:42:40.499516 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:40.499480    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zch9j_b668da5e-cfe9-47fa-9f63-16cdea56c0bc/discovery/0.log"
Nov 04 03:42:40.658980 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:40.658956    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54497bf445-q5p8t_189d27e4-0565-4c01-836a-ed99a907f34c/kube-auth-proxy/0.log"
Nov 04 03:42:41.095982 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:41.095952    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-74hkj_4afac37a-d227-4dd0-8f7d-b7c276c4974c/istio-proxy/0.log"
Nov 04 03:42:42.005005 ip-10-0-128-201 kubenswrapper[6019]: I1104 03:42:42.004972    6019 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-hk6mk_2030eefc-7cef-4652-97ae-f3fa1e560e13/istio-proxy/0.log"